Hi, guys.
Welcome back.
So in this section, we'll take a look at using Big Transfer, which is a transfer learning paradigm
or recipe book for increase for classifying images.
So it's very, very useful technique, I'd say, to create your own classifiers that get excellent performance.
So let's get started.
So open notebook sixty two and begin to lesson.
So once again, I'll just point out that this came from the official Carris tutorial site, and the
author is say a lot and will begin.
So firstly, let me just explain to you what big transfer is known as byte, similar to how vision transformers
have VAT.
But basically, it just it's just a way we can slow it pre-training that works on big data sets.
And these pre-trip networks are like things like a resonant 50 or resonant 150 to a pretty deep and
wide resonant.
And basically, it allows us to just load that network as a feature extractor and then train it on our
own data set and get some very good results.
And you can see VAT, which is in blue, has gotten better results than the best state of the art networks
here, at least generalist.
State of the art networks.
Not fine grained classifiers that might give better results, but you can see we're getting very good
results using this paradigm.
So let's get started.
So let's run the setup here.
This takes about three seconds right now.
Then we lowered the flower flows dataset, which takes about eight seconds as well, and we can visualize
the researcher seeing some images of the flowers in the dataset.
And the flowers dataset consists of 17 image classes, by the way, just in case you didn't know.
Next, we need to set some parameters, so we resizing the images to create a forward is actually fairly
large, but then we crop to 224 afterwards and send that image that size anyway.
And then we just said, these are the parameters here.
Actually, this is the different flowers in the sorry.
So this one is five clusters, not 17, like the original one.
Maybe we just made this one, maybe a subset of that one, and we just set some schedule links as well
as schedule boundaries.
So and what these settings really are, you can take a look at them here.
They're hyper parameters that Google has mentioned in their publication and blog post story.
It depends on the augmentation mix of these with very nice data augmentation technique.
However, you don't always need it depending on these parameters here.
Dataset size and schedule linked.
So you can see you can set these accordingly to what you want.
So that's 500 right now, and the boundaries are two, three and 400.
So we just passed those parameters here and we create a pre-processed function.
We created a pre-processed test, image function when we're inferencing images and then we just create
some repeat count using some of the schedule linked type of parameters.
Next, we define our data pipeline.
So let's run that there.
And now we can just visualize some of the training samples here.
Let's take a look from our pre-processing pipeline just to make sure it works.
And you can see it does.
So now we can move on to loading our network.
So we're going to load this resonant 50 model here and the Vati model.
So let's do it using it as a feature extractor.
Oh, sorry, I've already done this.
I believe so.
That's why it's running here.
So let's keep going.
So now we actually create our bit class here.
So this is where we just put a number of classes here, create our own then slate on top of it and then
get to put the VAD model as basically what we in the beginning.
So we get the features out of it and then change the head of network and we get back on there.
Next, we need to defined optimizer and losses so we can see we used to schedule boundaries as a parameter
for the boundaries here, and we're using something called peace waves constant to Q, which is not
optimized that we have we have come across before.
This is probably one that's specific to using B schedule boundary type of parameters and or sorry,
this is actually it schedules.
So it's not producing SD as a regular optimizer here.
Apologies for that.
Some learning, Richard, you can see here as well.
Let me just define, I guess, it's per schedule boundary as well, because legal boundaries do seem
to be changing the image size, I believe.
So now we're ready to compile the model, create or callbacks early stopping.
And now we trained a model, and you can see, even though we said that 40 bucks a train, it stops
quite quickly.
And that's because we have the early stopping parameters set.
And you can see we get ninety seven percent accuracy, which is really, really good, to be honest.
And now we can just plop the history here.
You can see we quickly got some good results out of it.
And that's because of its pre-trained on image net, too, which is very applicable to a flawless dataset
because of all the plant categories inside of it.
And we evaluated the model assuming with ninety seven point four or five percent.
And that's it for this lesson, and you can see the conclusion here that VAT is actually performed better
in a lot of these common like so far 10 data task and you can see it.
And basically, it's quite good.
You can consider it here.
So I would encourage you to experiment.
You can see the general state of the art and be when flowers is ninety seven point seven.
They're able to get that nine point sixty three.
This isn't using this model of this model.
Give us ninety seven point forty five, which is pretty good.
So it's very comparable to the state of the art in normal CNN model training from scratch.
So I assume it's from scratch who could be pre-trained?
But this big transfer framework or recipe you can call it allows us to get very, very high accuracy,
very easily.
So that's it for this lesson, and I'll see you in the next one.
Thank you for watching.