Hey.
Welcome back to the course in this section, we'll take a look at the all important technique of transfer
learning why you don't always needed training your left from scratch.
So let's begin.
So what is transfer learning?
Well, in practice?
Very few people train an entire CNN from scratch.
That that means a random initialization of which we don't always need to settle with randomly when training,
because honestly, it's relatively rare to find a dataset that is sufficient size to train these very
big networks.
So instead, what we do, it's common to take a pre-trained network that was trained in a very large
dataset something like image net, which contains 1.2 million images with towers in different categories.
And then use that as to use the weights from image net as the initial weights for all of the network.
So we take a network that has been trained on image net and then we can do several things.
We can apply transfer learning, we can apply fine tuning as well as we can apply it as a fixed feature
extractor.
Those are the three major scenarios of transfer learning.
So let's take a look at this.
So let's look at the rationale behind transfer learning and the way it works.
So training extensive models on vast image datasets like image nets can take weeks.
OK?
However, a model trained on so much data is going to have some very useful embeddings.
That's that's basically the features those filters learn is going to be applicable to many of the different
sectors because imagine you've created on a thousand different image classes.
Features that are trained on the features that have been left on those classes can be transferred to
other tests.
Example with image that has categories like tigers, lions and horses.
And then it can be useful, given that it can detect those features here and in many other mammals as
well.
It can be used to detect and distinguish between other four legged mammals.
If you have a data class with, say, different types of deer or buffalo as well.
So that's pretty cool.
Also, transfer learning is the concept where we utilize already trained models in other domains so
effectively that a model has been trained on image that is now applicable to achieve very good accuracy
and much faster training times in other sectors with other datasets.
So there are two major types of transmitting.
The first is feature extraction.
The second is called fine tuning, and they're both very similar.
However, there's a distinct difference which we'll discuss, and a lot of people actually confuse them
quite a bit.
And it's not that important to know the exact difference between them because most times researchers
are effectively doing basically doing the same thing just in different ways.
But I'll explain it shortly.
So what's how do we use a pre-trained CNN as a feature extractor?
Firstly, let's imagine this is a very deep sea into something like a resonant one to one, and it's
been trained on image net.
So these weights on these filters here are quite valuable.
These filters have learned so many different patterns and split different sports of detectors, color
blobs, all sorts of patterns.
It's little wonder that housing image classes.
So this is a quite valuable to us, these layers here.
So what do we do when we're using this and transfer learning?
Well, since these features are so valuable, valuable these weights, we freeze them, freezing them,
meaning that we don't trip, we don't train them anymore.
We keep those exact weights.
All right.
But what we do train is to fully connected layers here.
And that's because we're using this imaging that data sets on our new classes.
So let's imagine that we have with Tekin and pre-trained imaging that network that opens to 2000 classes
and we replace the head of the network.
That's a fully connected layers here, unchanged with outputs from a thousand to just three panda,
cat and dog.
And we used the widths from image net to assemble a feature extraction instructors.
And the only thing that's different now is the fully connected layer of a head with chants.
And oh, let's go over the steps of transfer learning by feature extraction.
So imagine this is only at work here where we have a pre-trained model.
This is a convolutional layers here, and this is the fully connected head.
That's trainable.
That's what we do.
So let's go over these steps.
So firstly, as I said, we freeze the bottom layers of a pre-trained that we have that so convolutional
layers in most cases, then we replace the top half.
Some actually have the bit of the top piece of the network with our own top.
So he just creates a few layers of fully connected nodes, set out one node number of nodes and definitely
set the number of output nodes to be the number of classes in our new dataset.
So if you're training on a data set that.
As cats, dogs and pandas, we said they would put the tree as opposed to what it would have been before
an image net, which is a thousand.
And then we just trained the model, trained this model here with these weights being frozen.
And we're just training the top players alone.
So now let's take a look at fine tuning the other method of transfer learning.
So immediately from this diagram, you can tell there's something a bit different now you can tell what
we're doing here is that we're keeping some of the weeds here frozen and making some of them trainable
here.
So let's take a look.
That's exactly what we're doing.
So the intuition behind a fine tuning here is that instead of training and sort of keeping all of the
convolutional filters frozen, we leave the lower filters frozen, but make them more upper end convolutional
filters.
We unfreeze those and we train those itself.
That's basically what fine tuning is, and the reason we all to call it fine tuning is that we also
use a very small learning rates here because we don't want to make big changes.
So like I said, intuition behind this is that these lower filters here early in the network, they
tend to learn generic features, things like edge detectors, simple patterns and all of those things.
However, the higher layers learn more intricate patterns.
So you're going to want to actually unfroze unfreeze those so that you can actually generalize and learn
the stuff in your new dataset so that you get a very good performing network.
So when do we use transfer learning?
So when is the ideal time to use transfer learning?
Well, the ideal scenario is if you have a very large new dataset and it's somewhat similar to the original
dataset, that's imaging that in most cases here.
And in that case, your model should not over fit and you should get pretty good performance.
Less ideal but still recommended is a few new data set is large but slightly different, or a bit a
bit different.
In that case, I would recommend you use fine tuning his well instead of the feature extraction method
of transfer learning next year.
When you need a small, even if it's similar to the image, not dataset or whatever pre-trained dataset
you use.
It's not always ideal for transplanting or fine tuning.
However, a useful idea is to often just extract the CNN outputs and then instead of putting the head
as a neural network, the fully connected layers use a linear classifier, something like a logistic
regression or an SVM, which we will be doing in one of our projects shortly.
Use that so your model gets better generalized generalization performance.
So some more advice here always use more lending rates, especially when doing fine tuning.
You don't want to change those valuable image net pre-trained weights too much because I mean, your
GPU is someone else's.
GPUs have spent so much time getting to his good with, so don't often mess that up.
So use small learning rates to not to so that you don't change just because it's too much and also due
to parameter sharing.
You can actually use different input size images when pre-training, and it doesn't affect your model
very much.
In fact, it doesn't effective it at all.
So what we'll do now will actually go back into the collapse of the code, and I'll teach you how to
use how to do transfer learning and fine tuning in both Keras and PyTorch.
So stay tuned for that take.