Hi and welcome to another introduction to regularization technique called early stopping.
This is a very easy one to implement and understand, as I said previously.
So the overturning problem, so generally at some point during a training process, let's say we set
the box at 100 at some point, maybe 40, 50, 60 epochs.
Our validation laws is going to stagnate or stop start decreasing or even sometimes start to increase,
which is not a good thing.
It means that it's over fitting on the training data set at that point.
So at that point, you really should stop treating your wasting time, wasting computational resources,
wasting electricity, contributing to climate change at that point.
So we need to find a way to stop at that point.
So how do we do that?
Well, basically what we're doing here this this technique where we stop training early is called early
stopping.
It's quite simple.
So here's how it's implemented.
So this is the diagram here shows a typical validation error and training error loss.
You can see training error continuously goes down because if we give it so many books to train, it's
going to keep getting better and better when the training dataset.
However, at some point it's going to stop getting better and the validation set of data this point
here key minus p kidswear to stop.
Let's stops here initially, and P is where basically it's stopped training and stop producing volition
error.
This is where we want to stop.
Basically, it's simply area where we want to stop here, and it's a simple algorithm in telling this.
Telling your model when to stop training.
How it works is basically we just have a simple conditional loop if you want to manually implement it
in your training process.
That says if our validation loss has stopped decreasing after X number of epochs, now let's just stop
the model and see if the we'd see a saved the best tweets where it was, Lois.
That's all it is and it is stopping is basically it's a very simple way to reduce overfitting, as I
said, because at that point, it stops a model from overfitting on the training dataset and it can
be manually implemented and paid too much because as you saw in the PyTorch, could we can we manually
treating it on every epoch?
You can actually see the code breaking down as we go forward back, propagate all of those things.
So we have easy access to obtaining the losses, data validation losses so we can easily test fitsum
code and data check and see if it's stopped decreasing, then stop training.
Or there's also which is not commonly available and commonly known, but PyTorch does have some API.
High level libraries like Lightning is one of them which will be living in discourse that allow us to
implement these features quite easily because something is actually very easy to implement in.
It doesn't take much work.
It uses something called callbacks, and we can do a number of things with callbacks such as saving
the best model, early stopping and a bunch of other things.
So it's easy to implement.
It's not not a big concern you should have right now.
So that's it for this lesson.
Just to reiterate, it is something that allows it to not put, not over fits on the training data,
which means that when overfitting you're learning patterns of noise in your training data.
Next, we'll take a look at batch normalization.
So thank you, and I'll see you the next lesson.