In this lecture we are going to discuss how to improve our model using a process known as data augmentation.
Consider this picture of a cat.
You and I can agree that it's a cat now consider this other picture of a cat.
It's pretty obvious that this is the same cat.
Unfortunately while this is easy for us it's not so easy for a computer.
Now the neural network may be able to identify this because of convolution but perhaps there's a way
to help the neural network learn this translational and variance using data augmentation
as you may have heard one of the reasons.
Deep Learning has been so successful these days is that we have so much more data deep learning continues
to improve as it trains on more and more data.
While other machine learning algorithms seem to hit a plateau.
One disadvantage of tabular data is that you just have what you have.
If we're doing classification to see if you'll pass my math exam and I base that on how many hours you
study and how many hours you play video games that can only depend on the data I collected through my
survey.
I can't just make up new data which would necessarily include my own biases about student performance.
I can't invent a new student who studied this much in play video games this much and then decide if
he passed or failed.
That wouldn't make any sense
however.
Images are an important class of data we can see them.
We know that both these images are cats.
I know that if I rotate this cat a little bit it's still a cat.
I know that if I flip a car upside down or left to right it's still a car.
Although you should probably be worried if you encounter an upside down car.
In other words what I'm trying to say is with images it is ok to invent new data.
Now here's one problem with this approach.
Data takes up space so the more new data I invent the more space it takes up and on top of that there
are an endless number of ways I can invent new data.
I can shift the image to the left one pixel two pixels three pixels four pixels five pixels and so forth
and I can also go in the other direction.
I can also rotate one degree two degrees three degrees and so on.
Do I really want to store all these combinations on my hard drive.
Probably not.
Also Google collab probably wouldn't have space for that anyway.
So what can you do
luckily none of this work is work you have to do yourself.
Have you ever considered how to rotate an image.
Well it's not as straightforward as it sounds.
Instead we can use the torch vision API to do all this work for us.
To understand how it's going to work we have to discuss the concept of generators and integrators for
the purpose of this lecture.
We won't really differentiate between the two
consider what you do when you want to write a for loop using indices 0 up to let's say 10.
We would do.
For i in range 10.
Now in python 2 you would not want to use range 10 because that would return an actual list of numbers
from zero up to 9.
Instead you would use X range which does not create a list but rather generates those numbers on the
fly.
That's why in Python 3 This is the default behavior for the range function.
If you print out range 10 you will not see a list of numbers from 0 up to 9.
As you might expect conceptually that's what it is but physically that's not what it is.
Instead it just says range 010 which is kind of not telling us much.
The range is a range.
Well it appears to be some kind of object on which you can call instance methods and so forth
like you might not have known is that you can actually write your own generators.
For example suppose you wanted to write a loop where the loop variable was a randomly generated number.
So for x in my random generator would be a loop where on each iteration of the loop X would be a random
number.
Can we do this in Python
The answer is yes we can accomplish exactly this using the yield command.
So here is how you would write my random generator function that generates 10 random numbers on the
fly as you can see the code to generate the entire list of values does not have to be computed beforehand
and more importantly the entire list of values does not have to be stored in memory.
Instead this takes up a constant amount of memory because as soon as the variable for each iteration
is computed and used it is no longer needed.
So in PI towards when we want to do data augmentation it's a similar concept instead of pre calculating
all of our augmented images beforehand we are going to create them on the fly.
You can imagine that it looks something like this for each batch of data we augment X batch using some
randomly generated values then we yield x batch and Y batch.
In other words only a small slice of X train is ever augmented.
On each iteration.
More importantly it's done randomly so you get something different each time which helps with generalization
what you should notice is that this looks curiously like our data loader which we have already used
for generating batches.
When we do Batch gradient descent
so how does this actually work in PI torch.
What we've learned so far in this lecture is just conceptual.
Practically speaking the answer lies in the functionality we have already been making use of specifically
you know that when we load in the CFR dataset it's just a function and this function accepts as input
an argument called transform.
At this point you should ask yourself.
I wonder what this does.
In addition as you may recall we learned in the last section the data loader object yields your data
set in batches as you loop over it applying the transform function on the fly
so to be explicit.
Well we have to do is modify the transform argument and instruct torch vision on how to augment the
data specifically torch vision comes with a whole collection of different transformations you can apply
such as slightly changing the colour cropping flipping both horizontally and vertically rotating and
so forth.
I like the idea of changing color because I don't think people mentioned it that often yet it seems
very important.
You would want your model to perform well under different lighting conditions and different colors.
I would recommend checking out the documentation and experimenting with each of them in different combinations
to get a feel for what has the most impact on your particular dataset.
As I mentioned early on in this course page which gives us a lot of building blocks from which we can
compose more complex things just like how we can compose complex neural networks from simple layers.
We can also compose complex transformations made up from these basic ones.
All we have to do is wrap them in a torch vision that transforms that compose object and pass this into
the transform argument when we load in the data
now because all of the data augmentation transformations are specified in the data function itself.
That means everything that happens after that does not need to change specifically consider what the
data loader would look like exactly the same as before.
All the instructions for transforming the data belong to the data set object itself not the data loader.
If you recall the only thing that the data loader does is call the transform function on the data set.
Similarly we should expect that the training loop also does not change.
We still have a nested for loop to do batch gradient descent.
The first loop is still for Iron Range epochs and the second loop is still for inputs targets in data
loader.
The only thing to be careful about is that while you do want to augment the training set you probably
don't want to augment the test set otherwise you get a different answer each time you evaluate it which
is undesirable
after all this.
You might still have one remaining question.
So far we've loaded all our image data from pre-built functions in torch vision but what if we have
our own image dataset.
Sure I can load images into an empire res using a library like pillow but once I've done that how can
I apply these transformations that torch vision provides.
It seems like in order to use them I must have a torch of vision data set object.
Luckily we will cover this later in the course but for now you'll just have to wait and see.