So in this lesson, what we're going to do, we're going to load the CNN amnesty model that we've trained
in the previous lesson and what we'll do, we'll actually analyze its performance.
We'll take a look and see which classes it's misclassifying and examine the confusion and classification
reports.
So let's open this notebook here.
Not arbitrary.
And you can see it's split into four blocks.
First book is where we at our previous three and model and our data.
So let's go ahead and do that.
And I've uploaded that model that we trained in the previous lesson to my Google Google Drive.
So now I can access it and download it quickly again within this notebook right here.
Next, we need to load our data set so we load of data said like we did previously.
I'm sorry, we load of model.
Sorry for that.
Like that was downloaded here.
That's end this simple scene and model that we've trained.
So let's load this model.
It's quite simple to just using the load model function from Keros, and that's done now.
We can load our amnesty to set.
So to do this, we import the M.
This object here from TensorFlow Dot or data sets we use include data function inside of it to get our
training data or training labels or test data and or test labels here.
So let's run this code.
There we go.
So we have we have a data set now.
So firstly, the first important thing we're going to do is taking a look at the classes or the input
images that our model misclassified.
So to do that, we basically first have to get our predictions out of our model.
So that's simple.
We've done this before in the adolescent.
What?
I'll do it again, just to be sure.
So we take our next test, that's the test data.
We reshape it by adding on this extra dimension one here, 10000 samples.
We know this previously, but you can always examine if you want to make sure by printing out its desktop
shape, which we do here, actually.
But this is that's printed out before and after.
So you can see the difference that how to reshape function does.
And by using Npr.org. Macs to get to the class with the best probability, by using models or predict
X tests as input, we get our predictions right here.
Stored in this variable 10000 values are stored in a variable and you can see the shape difference here.
We have ten thousand twenty at 28 and 28 by 28 images, and we have the same thing ten thousand twenty
eight by 28 images with the appended dimension here.
So we're now going to do this line is actually a bit irrelevant.
Actually, no, it's not.
Sorry, it's predicting the classes for ten thousand images.
That's fine.
So now what do we do here?
Well, we want to find a way to get which which classes of model misclassified.
We can do that by using number here.
So by getting the absolute value here, if we minus the whiteness minus a prediction, remember if it
was white us for a certain class like this in first class whiteness was five and the prediction was
five, it would be a zero.
That's that's fine.
However, when there's a difference like that C, it predicted a six instead of a five.
We'll get a one here and that this is this is no returns, basically a Boolean where we have zeros and
the positive digits because it's an absolute function here.
So we can use that known that stores, we still have the new results and then we use that using NP or
non-zero.
So we just check for all the non-zero values said.
This gives us to bully and I should say, and now we have the misclassified indices here.
So by printing, running this and printing this out, we can see all of these indices here were misclassified.
So the it's technically the ninth index in the test dataset was misclassified the attitude to 60 second,
90 second and so on.
Consider a number of misclassified ones here.
In fact, let's actually print number the length of that.
So we can see sorry.
Run it again and you'll see linked boats one.
And that's because this is actually an array within a very sorry, but so it's 506 values, it got wrong.
So out of 10000 predictions, it got 500 and six wrong.
So now let's take a look at visualizing the actual images that our model in this classified.
This is a very useful technique in diagnosing what your model is doing wrong because also sometimes
they can have mislabeled the actual data.
So it's always a good way to assess what your model is predicting wrong by visually comparing the predictions
to the actual inputs.
So let's take a look.
So this function, his just initial function, it's nothing special.
He would have seen this before in the previous lessons.
And this here's a block of code that plots the input image against a prediction so we can see them side
by side.
So let's run this block of code.
We're going to do 10 random the 10, the 10 random, I think 10 of the first, yes, 10 of the first
two misclassified indices here.
So we're going to predict we're going to get the predictions and the images for the first 10 of these
values.
Here is indices who indexes.
So you can see here this one was a five hour model, predicted a six.
Understandably so, though it does seem to be more four five than the six to me.
This one is definitely wrong.
This one is a four letter, predicted a six.
This one is a nine.
It predicted four.
Again, a nine mistaken for four.
This one looks like a two and predicted the nine.
So you can see the model isn't doing too well.
This one is clearly and it predicted a seven.
This one's clearly a nine and predicted it.
This one is clearly a four and I predicted of two.
So you can see at ninety five percent accuracy on model still isn't doing quite well on these classes.
So we need to improve this.
But that's for a little lesson for no.
Let's take a look at another more elegant way to plot this.
So by using some of my plot lib functions, we can just basically extract the values here.
And there's a mistake in this code only to fix this because the lesson here, I don't fix this mistake.
Sorry about that guy.
So, yes, a fixed a mistake and what we're going to do.
We're going to just print some subplots here, five by five subplots where we just use a title to show
a prediction of the class and the original class and then show the image below.
So let's run this code and take a look at the outputs.
There we go.
So this is a nicer way to actually visualize it as well, and you actually get the actual ground shoot
levels and the predicted levels here as well.
So you don't actually have to infer it or view them side by side.
And as you can see, the model is doing not too good on these losses, is it?
Well, these are obviously will obviously have the impression it's not performing too well because these
are the classes we got wrong.
We're not looking at the classes that got it right.
Otherwise everything would look fine.
But it's good to analyze this so we can see a model definitely does have some weaknesses here.
So we'll stop there.
And next, we'll take a look at creating our confusion matrix.
So I'll see you in the next video lesson.
Thank you.