Hi and welcome back to the course in this section, we'll take a look at something called callbacks,
which allows us to implement early stopping, check pointing, learning, read schedules and much more.
They're available in broad Keras and PyTorch.
However, it's much easier to implement and carries it away.
But let's begin so callbacks callbacks are used to perform different actions at different stages in
the training.
But why is this useful?
Well, what if we wanted to see one model after the end of each epoch?
Or what if you create a model for 100 epochs?
But after Tuti so that all of a fit?
Wouldn't you want a way to monitor training so that you can stop trading and stop wasting time at that
point?
Or what if we wanted to log information somewhere during training that can be analyzed elsewhere?
Well, callbacks allow all of those things.
They are the solution.
They can be used to perform a number of things, such as early stopping, which we discussed in our
regularization section model, check pointing, learning, read, scheduler logging and remote monitoring,
as well as a custom function.
So let's take a look at this graph here.
You can see this is an example of overfitting model, and you would want the model to stop here.
This is what early stopping does.
It allows us during the training process.
It allows us to if for validation, los stagnates or stops decreasing and actually can start to increase.
It allows us to do set something called early stopping criteria that allows our model to stop trading
at that point and ends the training process so you don't waste time.
What about model check pointing?
Well, this is how you set the parameters and carrots for model check point.
But what exactly is this?
Well, models are pointing allows us during training to periodically save the weights after each epoch.
That's very good because as you saw in earlier stopping, sometimes the best model isn't the one that
you trained after the most ebooks.
In fact, it could be one of the early models that just got it got to be its right that give us a very
good loss.
So you want to actually store we to monitor the validation loss, and you can see if the best model
of, well, trading so you can train for a thousand epochs if you had all the time in the world and
powerful GPUs, and it would save the best model out of all of those ebooks using the models are pointing,
which is quite cool.
So it's also allows us to resume trading in the event of a crash, and the checkpoint actually contains
the model of weights and the model itself.
So when you see if all those checkpoints that are light allow you to resume training or whatever you
want to do, all those all the checkpoints are basically the model weights at a time, and this is how
it can be configured.
So now let's take a look at lifting weights scheduler.
So this is what it looks like for a living which has Shatila.
But what exactly is it?
Well, we can avoid having a little loss oscillate around a global minimum, which is not what you want.
You wants to find a global minimum.
And we do that by attempting to reduce the learning rate by a specified amount.
So if no improvement is seen, you know, a monitored metric, which is a validation list, typically
we wait a certain number of epochs.
That's the patient setting, which you see here and then does call back and then it reduces learning
rate by a factor that is set here.
Point two.
This is a minimum delta can change, so you can see by using this this function called reduced learning
rate on pilot to just quite descriptive of what it does if you're living, if your loss stops decreasing.
This automatically reduces the learning rate, so it's quite dynamic and quite useful.
You can also do something like logging like this.
You can automatically log your models training stats, using checkpoints, tens of what checkpoints
here.
The stores stausholm model results in the directory and dot logs directory, and you can just load a
tentacle.
Is doing these commands in your Pi Python book and it brings up this.
This brings up this nice web UI where you can actually track and monitor your model as a tree and so
you don't get the graphs afterwards.
You can look at the graphs life, which is quite cool.
I will be doing that in all two models that we will be treating in club.
So stay tuned for that.
So that's it for this section next to what we'll do.
Well, actually look at callbacks in the court and Keras and PyTorch.
So stay tuned for that.
Thank you.