In this lecture we are going to look at a code lab notebook that does transfer learning with data augmentation.
This lecture is going to walk you through a prepared code lab notebook although a very good exercise
which I always recommend is once you know how this is done to try and recreate it yourself with as few
references as possible.
As usual you can look at the title of the notebook to determine what notebook we are currently looking
at.
It's called Pi torch transfer learning with data augmentation
so at the top we have all our usual imports.
We have one extra special thing which comes from the torch vision library and that's the model's module.
This is of course where pre trained image networks are accessible from.
We have some other minor new imports as well but you'll see how those are used as we go along.
Next we download the data which is called the Food five k data set.
This dataset is a set of images of food and non-food items so it's a binary classification dataset and
your job is to classify whether an image is food or not food.
The next step is to unzip the file since there are a lot of files we're going to use the Q Q option
to suppress the output.
Next we're going to do an LSD to see what got unzipped.
As you can see there are a few new folders here specifically evaluation training and validation.
So let's check and see what's in the training folder.
So as you can see it's just a bunch of image files.
So what do we know right away.
Well we can see that this is not in the format we need in order to use the image folder data set object.
What we are looking for is one folder for each class of images what it looks like though is that all
the images for one class start with a zero in the file name whereas all the images for the other class
start with a one.
Of course we shouldn't try to guess whether that's true or not we should at least do a sanity check
to make sure that this is actually true
so let's print an image that starts with zero.
Just to be sure.
So I've chosen 0 8 0 8 but you should try other images yourself too.
All right so this is a flower which is not food
next let's try an image that starts with one I've chosen a 1 6 1 6.
As you can see this appears to be food there looks like pizza.
Again try a few other images yourself to see if all the zeros are non-food and other ones or food
since these images are not organized the way we need.
We're going to have to move the files around a little bit.
So let's start by making a new folder called data inside this folder.
We're going to have two more folders train and test.
Now the original dataset came with three folders evaluation and training and validation but we don't
have any need for three separate datasets.
So what we're going to do is just use the training folder as the train set and the validation folder
as the test or validation set will ignore that evaluation folder inside the train and test folders.
We need to have a folder for each class so you can see here that I've created two folders inside each
of the train and test folders food and non-food
next.
It's just a matter of moving the images moving is better than copying in this case since we don't want
to take up too much space and waste time replicating the data on disk so you can see here that I'm taking
all the files that start with zero and end in JPEG and moving these to the corresponding non-food folders.
I'm taking all the files that start with the one and end in JPEG and moving these to the corresponding
food folders
next I'm going to specify the transformations to perform on both the train and test sets.
As usual you should try to play around with these data augmentation parameters to see what effects they
may have if any.
You'll notice that the final transformation I apply happens to both the train and test sets.
It's a normalize which as you might expect does normalization and I've specified what appears to be
just a bunch of random numbers.
So what are these random numbers and why are they there.
Well you have to remember that these are pre train networks with very specific weights.
So we can't just pass in any image in any way we want into this neural network.
We have to transform our image in the same way that the original authors of the neuron that did.
Otherwise our input won't make any sense.
So this is more or less like that.
There are some additional details that I don't want to get into now you'll notice that there are many
more transformations we're applying to the test set which may seem odd you might think why are we applying
data augmentation to the test set.
And actually we're not.
So these transformations are static transformations they don't augment the data they're just necessary
for the neural network to work.
So the first transformation which of course has to be applied to both datasets is the resize.
CNN's work on image batches of constant size.
So first we'll resize the 256 by 256.
After that we cropped the image to be of size 2 to 4 by 2 to 4.
OK.
So that's why we have some transformations on the test set.
Now you notice that the train data has a few additional steps in between.
So random rotation color jitter and so forth.
As mentioned you should feel free to play around with these.
Change the order and so on
next we create our image folder datasets.
This is pretty trivial.
We pass in the paths to the image files and the transforms that we just created.
Next we create our data load of objects.
As usual
next we define our model.
So this is going to look very different from what you're used to.
First we grab the VEGF 16 model which is located in the torch of vision models module we set pre trained
equal to true because we want pre trained weights and not random weights.
Next we freeze the weights so that these weights are not updated during training.
We can accomplish this simply by looping over the model parameters one by one and sending the requires
grad attribute to false.
Next we print the model so you can see that it's a very large model.
Importantly though it has a structure we've seen before so the first part is a sequential.
That wraps all the convolution and pulling layers as you can see this is attached to an attribute called
features.
Next we have an average pool 2D.
And finally we have another sequential that wraps the dense layers and this is attached to an attribute
called the classifier
so if we print out model that classifier we see just the end and part of the CNN.
This is the part we want to get rid of because we want to attach our own head
so next we're going to get the number of features that are output from the first part of the model and
call that end features.
We need this because it's going to be the number of inputs into our head so we can get this by calling
classifier of 0 which gives us the first layer and then calling the attribute in features.
So we have twenty five thousand eighty eight features.
Next we're going to create our own classifier which will just be a regular linear layer logistic regression
and we'll assign this to the classifier attribute to replace the old one so if we print the model again
we can see that the classifier attribute now refers to our own linear layer
and actually move the model to the GP you as usual
actually after this everything is the same as before.
So we create a loss and optimizer we create the training function we call the training function we look
at the last for iteration and we calculate the accuracy
and these are the results so we get ninety nine percent accuracy on this large image dataset using transfer
learning.