Hi, guys, welcome back.
What we're going to do in this section is we're going to implement a rank one and rank five algorithm
to get our accuracy metrics based on the rank one around five.
Hi, guys, welcome back in this section, we'll take a look at using PyTorch to implement the rank
one or rank five accuracy.
Technically, we're making an algorithm that can give us a rank and accuracy.
So let's get to it.
So open this notebook here, which I've already done and what we're going to do.
We're going to load of 16 network and we're going to use the same images that we did before, but now
use them in a way to calculate to rank one or rank five or rank 10 accuracy based on on the ground charts.
So it's like we're testing it that that was images become a test data set now.
So there we go, so the model is loaded, and again, we'll have to do a normalization transforms like
you've seen before.
Set the mood to evaluation mode.
Download our test images here and image classes.
Image cast net image net classes file that gives us of a class names.
Then we'll import all PyTorch modules, as well as ghetto class names, file variable from the image
nets classes, JSON file.
And now we'll just to implement this this run on a single image and get the output.
So this is a burrito and it predicted a burrito, which is quite cool.
And you can see this is the steps we took to convert this image here.
So this is the end.
We point the image a little bit to the image, the path for the image.
Then we just converted to Tensor by doing all of these operations so that we can then finally feed it
into the model to get the output.
Then we get the maximum clucked index of the maximum probability class and then use that index to look
up the class name and then we print the class name on the title of the image.
So it predicted a burrito.
Quite cool, isn't it?
So let's take a look at something we need to get a class probabilities because remember rank one or
rank and basically we need to get the rank in the number of the top, cut the top probable categories
from the network.
So how do we do that?
So let's take a look at this.
So so what we're going to do, we're going to import this pay to its function called functional and
call this an nef here.
And we're going to apply that.
We're going to use that to get the soft max probabilities from the output.
That's basically all we're going to do because previously we just used ARG Max to get the top probability
index.
Now we're just going to use it to get the probabilities here.
So then what we do, we use probability topic here and we just specify how much probabilities we want,
said the dimensions to one.
Since we get it in a nice array and this returns, this this object that we get from the puts off max
function returns the top probability and top class, but it's not exactly it's many of them.
So let's see what we get here.
So you can see this is the output.
What this means here is that 0.87 is the highest probability that the class returned, that the number
of the model written sorry.
And this is a class.
This is nine six five corresponds to burrito.
Let's just double check and make sure.
So let's open this up and I'll scroll all the way down to nine six five.
I believe it was.
And yes, nine six five is burrito.
So let's take a look at that again.
So you can see this is the class it predicted for Burrito 965.
And this is a probability point eight seven.
Let's see what it gave this relatively low probability point zero one seven seven percent, up one 1.7
percent.
That is nine to four.
Let's take a look at it as well.
Guacamole.
That's that's kind of cool.
Actually detected guacamole.
I didn't see guacamole inside of this, but it's kind of weird that a category is quite similar.
It is possible, though.
This is because this is actually a good example of overtraining.
That the pictures of guacamole is that the network is trained on may have actually had burritos in them,
too, so it's understandable that the network could think of Burrito as a guacamole, but it doesn't
give it that much probability confidence here.
So let's look at 9:31 Bagel.
Kind of makes sense.
It looks more like a wrap than a bagel, but similar.
And nine three three cheeseburger, I guess, because it's a ground beef here and nine two three, which
is pleats, which which is OK.
I mean, it isn't actually even a platter, but maybe because it thinks it's food and the association
of these colors and these textures do go to kind of go hand-in-hand with pleats.
So those are the five top classes.
This model returns for this image, with the highest class being awarded to Burrito, which is 87 percent.
So that's quite good.
So now what we do?
We can put those top classes here.
That's the top class.
The index is here to get the indexes right.
Actually, I just set out to get the indexes right there so we can put it to an umpire and then index
it because this is a multi-dimensional array here.
So we just use zero to get the index to get the first elements in that area.
And that's this here.
So know what we do.
Let's create a class that gives us our class names.
It's quite simple to do.
So we just input something that gives us.
We just take this input.
Top classes here and we want to return the string of names here.
So it's quite sorry.
Quite simple to do that.
So what we do, we just take up classes, get the indexes here, which is what we did right here, and
then we just created an called all classes and then four top class and top classes.
We just give the class name out of our dictionary here and then appended to this array.
So now we have an array of the names.
As you can see, it works quite well.
Burrito, guacamole, bagel, cheeseburger plate.
So not only have that closeness now since we don't need it anymore, that's constructor function to
give us to rank in accuracy now.
So this function is actually quite simple.
So it's designers, call it Rankin.
It takes a model as the input directory of the images to ground truth, which is the correct class levels,
which is something you need to set prior in, which is how much which is to rank you want.
And we can show images tenant of yes or no, this something a true or false.
So you would have seen some of this code before.
What we do, we just dive into the directory, get all the file names of the editor directory there.
Then we just look through those images.
We open them.
We passed them to a network at the outputs, get the probability probabilities then used to get class
names, function to get to class names as well and store it into disarray.
And if the images, if we want to show images, we just plot them here.
This here being the same subplot plotting function that we have seen in previous lessons.
So what happens now is that we use to get school function that sorry for that, to get school function
that we created, we created below here.
All this function does just calculate an accuracy score based on that, all the top classes of returns
here.
So this is an array of five results or whatever and is and number of results that we set in here and
check to see if any of those end results are in the ground truth.
That's basically how we do it here.
Well, we did previously here if the ground truth already had the correct one.
So let's take a look at what the ground to is.
It's basketball, the German Shepherd, Limousin, blah blah blah.
So what happens now?
It basically looks at a first class.
Basketball is basketball that any of the top in classes here.
If yes, then it just increases the label count by one.
Otherwise, if not, it ignores it.
And then we just divide a total number of correct labels here.
This is a variable.
We keep track of the correct times.
It's in the end classes, it returns.
So it's a simple function and it works quite well.
And the only bad thing about this code is that you need to manually set what your ground should labels
and needs to be in the correct order that they are loaded in this area only forgery.
So what I would suggest you do is run this only file's array here.
If you were going to use your own images, run this line of code with the directory of your images,
get the order of the images that they are in and create your ground to the rear.
Based on that, and you can do it manually, or you can automate it with some CSS fees and maybe put
the class teams as to file names and extract it automatically so you don't have to do it manually.
It's just giving you some tips on what you can do.
So anyway, we're ready to run this code.
Let's get our top.
Let's go to Iraq five accuracy top five what I was going to see.
So you can see it's predicting it here.
And what this what this function is doing is that it's playing the the forget this warning here, but
it's just displaying the classes top five classes.
So we have basketball fears about a rugby ball, tennis ball, volleyball.
Similarly, German Shepherd, German Shepherd, Hello, Eskimo dog, husky, Siberian blah, blah blah.
So let's see what the final rank five accuracy of rigidity 16 is 88 percent on all those tiny little
test data set here of maybe eight nine images.
And now let's get the ground and one accuracy rank one accuracy, I should say, so we can see it's
going to it's going to definitely be less than 88 percent.
But let's see what it did.
77 percent.
Not too bad.
Now let's see what our rank then accuracy is.
Hopefully this is a hundred percent, so let's check it out.
Idiot PR. So whatever class it's getting wrong here, it's getting it very wrong, I believe it is this
one here.
No local matters, correct?
It is this one here.
It's definitely causing some problems.
So it's getting this one is not even getting big laughs in the top 10 predictions in this for this image.
So that's not great news.
But as you've seen previously, other networks do tend to get it right sometimes, especially the deeper,
bigger networks.
So the experiment with those, so we'll stop there and what I'm going to do, I'm going to use the same
algorithm would harass pre-trained models.
So let's take a look and see how we do that.