Hi and welcome back to the course in this section, we'll take a look at a new metric that's used commonly
for benchmarking classification models, and it's called the top one or top end.
What's up?
Five accuracy, also known as around one, around five.
So let's get started.
So what is this accuracy?
What is this metric?
Tell us?
Well, remember when we input this colleague, this dog here into the pre-trained network, it would
often sometimes written the wrong class, but returned.
But if we inspected the results, you can see this is the output here.
You can see sometimes that the actual correct class is maybe the second or third or fourth or fifth
ranked class.
That's interesting.
And as I said, here rank and is a way to give to classify as accuracy a bit more leeway.
So instead of it returning, Shetland Sheepdog has a particular class, which it would do naturally.
When we consider around inaccuracy, we look at the top five or top tree, whatever it is.
So let's assume and it's five here.
So if the correct class belongs in the top five highest probability classes that are output from a little
CNN, then we consider that as being correctly identified.
And that's why for rank five accuracy, you see the accuracy of scores go close to ninety nine point
eight percent in a minute.
However, if you look at the rank one or top one accuracy, it's usually at 80 percent roughly if the
state of the art model is sometimes 85 89 percent, depending on how good the model is.
So basically, that's it sort of rank, and it's basically considers the top in classes with the highest
probabilities and considers that as the predicted level.
So if any of these categories here are ground, should it considers that correct?
This is an example of the top one accuracy for image net and top nine percent up top five percent accuracy
for imaging that you can see.
This is the best, more efficient nutshell to model the second by efficient method piece by dash B-6
wide model.
The top one accuracy is a 90 percent, which is remarkably good for image net, by the way.
And you can see the top five accuracy with ninety eight point eight percent.
So that's also very, very good.
We're getting close to human level performance on the that.
I'm actually not sure what human level performance and imaging is.
There actually are probably some benchmarks out there that take a random group of humans and test them
on it.
However, with a thousand classes, it will be quite subjective sometimes for people to actually assess
those things.
Anyway, this is it for the top end or top one, top five or rank and rank one around five accuracy
metric.
So that's it.
And what we'll do now will go into the could into Keras and PyTorch, and I'll show you how we actually
can get the top and accuracies quite easily for models that we've from pre-trained models, actually.
So I'll see you in the coding lessons shortly.
But.