In the previous lesson, we saw how to do transfer learning.
But when we retrain the inception classifier features for cats versus dogs,
we ended up over-fitting again.
We also had augmentation, but despite that, we still suffered from over-fitting.
So let's discuss some ways that we can avoid that in this lesson.
Now here's the accuracy of our training set versus our validation set
over 100 epochs.
It's not very healthy.
There's another layer take in Keras called a dropout.
And the idea behind the dropout is that layers in a neural network can sometimes
end up having similar weights and possible impact each other leading to over-fitting.
With a big complex model like this, that's a risk.
So if you can imagine the dense layers can look a little bit like this.
By dropping some out, we make it look like this.
And that has the effect of neighbors not affecting each other too much and
potentially removing overfitting.
So how do we achieve this in code?
Well, here's our model definition from earlier.
And here's where we add the dropout.
The parameter is between 0 and 1 and it's the fraction of units to drop.
In this case, we're dropping out 20% of our neurons.
For comparison, here's the chart of training against accuracy
from before the dropout was added.
When you see validation diverging away from the training like this over time,
it's actually a great candidate to try using a dropout.
And here's the impact of the dropout.
You can see that it's very significant.
So lets take a look at this in the workbook.