To this point, we've been creating
convolutional neural networks that train to
recognize images in binary classes.
Horses or humans, cats or dogs.
They've worked quite well despite having
relatively small amounts of data to train on.
But we're at a risk of falling into a trap of
overconfidence caused by overfitting.
Namely, when the dataset is small,
we have relatively few examples and as a result,
we can have some mistakes in our classification.
You've probably heard us use the term overfitting
a lot and it's important to understand what that is.
Think of it as being very good at
spotting something from a limited dataset,
but getting confused when you see
something that doesn't match your expectations.
So for example, imagine that these are
the only shoes you've ever seen in your life.
Then, you learn that these are shoes
and this is what shoes look like.
So if I were to show you these,
you would recognize them as shoes even if they are
different sizes than what you would expect.
But if I were to show you this,
even though it's a shoe, you would
likely not recognize it as such.
In that scenario, you have
overfit in your understanding of what a shoe looks like.
You weren't flexible enough to see
this high-heel as a shoe because all of
your training and all of your experience in what
shoes look like are these hiking boots.
Now, this is a common problem in training classifiers,
particularly when you have limited data.
If you think about it, you would need
an infinite dataset to build a perfect classifier,
but that might take a little too long to train.
So in this lesson, I want to look at some tools that are
available to you to make
your smaller datasets more effective.
We'll start with a simple concept, augmentation.
When using convolutional neural networks,
we've been passing convolutions over
an image in order to learn particular features.
Maybe it's the pointy ears for cat,
two legs instead of four for human, that kind of thing.
Convolutions have been very good at spotting
these if they're clear and distinct in the image.
But if we could go further,
what if for example we could
transform the image of the cat so
that it could match other pictures of
cats where the ears are oriented differently?
So if the network was never trained
for an image of a cat reclining like this,
it may not recognize it.
If you don't have the data for a cat reclining,
then you could end up in an overfitting situation.
But if your images are fed into the training with
augmentation such as a rotation,
the feature might then be spotted,
even if you don't have a cat reclining,
your upright cat when rotated,
could end up looking the same.