So now let's talk about the all important lesson, which is assessing model performance, so we're going
to take a look at the metrics we use to assess model performance.
So let's do a quick recap on something here.
Remember, when we're training a model, you have to split your dataset into two or even three segments.
So generally, let's just think about it as two segments.
For now, even though this diagram has three, I will mention the two at one afterward and the explanation.
So you understand.
But remember, we're training a model with something data that's a training data.
The model sees that data and tries to learn from that data.
That's important.
That's basically what the model uses as its foundation for knowledge.
However, how do we know that the model hasn't only learned what the training data says and then can't
actually reproduce stuff in the real world with unseen data because you can have a model that performs
a very, very well in our training dataset.
However, when it sees new data, it performs absolutely poorly.
And that is called overfitting, by the way, which we'll talk about in later examples.
But for now, this is how we split.
Or do you have a training dataset?
You have a validation dataset.
The validation dataset is what you've seen, especially when training to Kerry's model because they
actually specifically state validation laws and validation accuracy after each epoch, after each,
after the data input data.
All of it has been passed through the network.
You didn't test the performance of your network on the validation data just to see how well it loose.
And that's basically the training process here.
So what about a test dataset?
Well, oftentimes you'll see people use validation and test interchangeably.
And I tend to do it as well.
Everyone tends to do it because most times you don't have to split your datasets into tree.
You can split it, split into two just to train and test generally what it's called.
And if you would testing that on a two dataset, that's a tiered, unseen dataset, separate set, which
we will call the test here.
That's basically a way to assess your final multiple model performance.
A lot of people who trained Kaggle models will be will be familiar with that concept where you can have
a training dataset, you can split that training with a setting to train and test, and then you have
your final performance test or validation data set at the end of the competition.
So let's take a look at example of the training process here.
So remember, we pass all of the images true over network forward propagate, then get the losses,
then use back propagation with gradient descent to optimize the weights.
That's basically what happens during one epoch, and then we test it on our validation dataset to get
lost loss accuracy to basically to see how it performs at that point.
Then we do this again for eyeball to eyeball tree epoch and all the way up to what epoch and it's off.
So that's when we end.
Typically, that could be 50 epochs, but it's up to you.
It's up to the network, although how much time you have, how good you want and how much time you have
to devote for your model training.
But generally, 50 is a good number to equate.
And then in the end, you can actually test that model that you've after.
You've trained here on an unseen test dataset to get the loss of accuracy.
That's the best practice when training deep learning networks.
So let's take a look at some of the basic performance metrics we use.
So what do we use to calculate what is back propagation?
Use back propagation uses our training loss.
That's how it updates to gradient weights.
Do we?
It's on the network as we use as we apply the chain rule downstream, the network gets right to left
on during training.
So we use the training loss for that.
So that's the first metric which you've seen before.
Training accuracy is the other metric that we tend to look at.
Treating accuracy just simply tells you how well your model is performing on the training dataset.
So how many of the training training categories that all model gets right after during the training
process, then we have a test or validation loss and test and validation accuracy as well.
So you can see these are all key metrics here.
So take a look at something here.
What if a were handwritten?
The digit model had issues with identifying its entries, so you can see this is the actual label here
and this predicted tree when it's actually it.
And in this case, here actually was also it, and it predicted a tree.
I mean, to be fair, I think.
Has a tree, but let's assume maybe the pen skipped over here and it was in it.
This looks more like an idiot, but it can easily be mistaken for a tree.
But either way, how do we identify such problems with our model?
How do you even know which classes are performing the worst, which are performing the best?
So how do you even know?
How do you identify this problem that your model is predicting trees inside of it?
Can we use the accuracy that we saw before?
Can we use these training metrics?
Well, no, we can't.
What we need is something called the confusion matrix and Classification Report to identify these sort
of intricate, intricate models, specific predispositions.
So in the next section, we'll take a look at both the confusion matrix and classification report.
This is a very important section in understanding how all models perform in the real world.
So go over this lesson again.
Go over the slides.
Watch the videos a couple of times just to make sure you understand these lessons.
Thank you.