In this section of the course we are going to cover one very important topic in a modern deep learning
which is called transfer learning.
This lecture will introduce you to both the intuition and implementation of transfer learning.
So by the end of this you'll be able to understand that the subsequent code.
The nice thing about transfer learning is it allows you to obtain very good off often the state of the
art results with less effort and less resources spent
the intuition behind transfer learning is this.
Let's start with the basic observation that in a CNN the features that are learned at each layer are
hierarchical.
In the beginning layers the CNN only learned some basic lines and strokes in the later layers.
The features get progressively more complex from lines and strokes to parts of faces eventually to full
faces.
This is of course if you train your CNN on images of faces.
You can also train your CNN on other types of images as well.
Of course
but here's another basic observation.
We recognize that the initial layers learn to recognize just simple lines and strokes is a plausible
that other task that don't involve looking at images of faces would still find a line and stroke features
useful.
The answer is yes because probably all pictures will have lines and strokes.
The difference between a car and a face being that the way that you put together those lines and strokes
will be different in this way.
We can imagine that the first few layers of a CNN might be useful for a wide variety of tasks from facial
recognition to vehicle classification to recognizing anomalies in medical images.
This is the basic idea behind transfer learning.
It's the idea that the features that I found for one task could be useful for some other task transfer
learning really took off in the field of computer vision so that's what we were going to focus on in
this course.
Probably the most widely known large scale image dataset is called image.
This is a dataset that contains millions of images and 1000 categories meaning it can recognize 1000
different objects.
Every year there is a contest to see who can build the best image classifier on this dataset.
The nice thing about the image that dataset is because it's such a large dataset containing so many
different kinds of images we can transfer the features learned from this dataset to a wide variety of
tasks suppose we wanted to build a dog versus cat classifier.
That's easy because imagine it already knows how to classify dogs and cats.
Suppose we wanted to build a car versus truck classifier.
That's also easy because image that already knows how to classify cars and trucks is even possible for
us to use the same features for things.
Image that has not seen before like say images from a microscope
all right.
So that's just the high level intuition.
Let's go deeper and discuss how this will actually work conceptually.
First we recognize that it's not feasible for us to train at CNN is on average than ourselves although
it's a lot less expensive and time consuming these days compared to the earlier days of deep learning
in the old days it was common for CNN training to take on the order of days weeks or even months.
Nowadays we have distributed training multi GP clusters and so forth but at the same time it's still
not really something you want to do on your own.
Luckily many of the major CNN ads that have won the Emmy that contest in the past have released their
pre trained CNN is to the public so there's no need for you to do any hyper parameter tuning.
Try out different learning rates and so forth.
All that has been done for you and the results are open source doubly awesome is that these pre train
networks are already included in PI torch.
We'll discuss what they are and how to get them later
let's say you found a super powerful state of the art CNN train on the image that data set and now you
want to use it to do transfer learning for your specific task.
How can you do that.
Well let's start with the picture.
This is a picture of a typical CNN.
It's a series of convolutions and pulling is followed by a series of dense layers.
One perspective on the CNN is that it's made up of two parts part one which involves convolutions and
pooling makeup a feature extractor or a feature Transformer.
It takes your raw image and converts it into a feature vector which is a useful representation for an
ends and other machine learning models.
Part 2 is just a feed for neural network or in a in which we learned about earlier in this course.
We can think of Part 1 as the body and part 2 as the head.
So here's how transfer learning works.
What we do is we chop off the head keep the body which is a useful feature transformer and then attach
our own head.
This head can be used for whatever task we want to perform for example identifying cars and trucks.
In that case we might use a logistic regression with a sigmoid output since as you recall the sigmoid
activation is the appropriate activation function for binary classification optionally you could add
more layers to make it a feed forward and then
here's another important detail about transfer learning.
There are a few approaches to this and I'm going to teach you about the most important and the most
basic version.
The idea is we're going to freeze the weights of the body and only train the head.
In other words only train the logistic regression layer.
You can imagine that if we use that gigantic pre train to CNN with 100 layers it would train very slowly
because we would have to update all the weights of a 100 layers CNN.
But by freezing the body we can simply ignore those weights and only train the weights of our own linear
model.
The important point is we don't need to train the convolution players because they have already been
trained on a data set much larger than anything we could have gathered ourselves.
The weights we already have for those convolutions are very good.
There is no need to try to make them better.
Instead we can assume that they give us good features and worry about training the top
one major advantage of using transfer learning is that you don't need a lot of data to build a good
model.
You might recognize this graph which basically says that the more data you have the better your deep
learning model is.
This is opposed to classic machine learning methods which tend to plateau after a certain point.
So you might think in order to build a good a deep learning model you must have a lot of data which
will in turn make your neuron that work take a long time to train but with transfer learning.
This work has been done for us.
We don't need a lot of data because the earlier features we're already trained on a lot of data more
data than we could have practically handled ourselves and so transfer learning allows us to get the
benefits of deep learning without needing to have a ton of data ourselves.
And on top of that it means we can train our model a lot faster.
Not only are we just training a logistic regression but we have a smaller dataset compared to what we
would need if we trained our model from scratch.