Hi and welcome back.
So in this section, we'll take a look at Grad Cam.
Remember, Grad Cam tries to help you understand where your CNN is looking when it makes its decision.
So let's open this notebook notebook 11 or already have it up.
So let's get started.
So what we're going to do, we're going to look at a few of these class activation map visualization
algorithms.
The first one we're going to do is basic grad cam, then grad cam plus plus that score cam and then
a faster score cam.
All of these allow us to see, well, CNN is effectively looking.
So we're going to use to have too many sessions open.
Let me just two minutes.
These.
All right.
So let's install this is the same visualization package carries this that we've used in the previous
lesson.
It's a very, very good visualization package.
I don't believe any exist.
Well, they may have some for PI to watch, but I don't think they're as good as this one.
So let's look at all libraries and print for using GPUs.
And in the meantime, we're just going to load our Keros pre-trained model.
The big pre-trained model.
It's not a carris model to say.
OK, so let's look at our Big 16 model this run, that block of code and a download model where it's
quite quickly because everything is stored on Google's backend and the internet connection is quite
quick.
So it's 128 million parameters.
It's quite big.
Now, let's download our test images.
These are the images that we'll be using to demonstrate the cloud cam algorithm.
So I've heard you run this, so because it can just check the images would be here.
So you download these here if I don't do it again to create more copies.
So I've done this now three times its way to spare it one bit, too.
So let's now visualize these images.
So we'll just use the regular so--but plots that we've done before and we plot the class title from
appear here we have stored and so we can see that this is the input goldfish to bear and also trifle.
So what we're going to do now, we're going to input these images into our into a great camera algorithm,
specifying that do we want that algorithm to look at that specific class because we need to know basically,
if we're identifying a goldfish, we need to know where CNN is looking when we went into identifying
a goldfish.
What features it's looking at.
So that's quite cool.
So to do that, we have to basically just create a lost function that basically gives the outputs of
the self max layer for that specific class of goldfish as class one.
The bill was class 294 qualities for that noise and assault rifles for one tree.
That's a class number.
So let's create this function and know create a model.
Modify the function which you've seen before.
These are where we have.
We put the activations to linear activations on the second to last live and now we can implement grad
cams.
Let's take a look at how we do that.
These are the functions we need to load in order to use graphical.
And this is important to note here from the TBTF Christmas package, we get crowd can and will do it
to create a calm function here.
And this takes the model as the input, the model modifier.
But we created a model.
Modify both here and we just don't clone the model in that case.
Next, we take this great calm object that we've created and we passed a loss to it.
We pass X, which is basically just a back end of Keros, don't have to think too much about that.
Is just used internally here.
And we passed have been in the penultimate layer, which is basically the labor force of MAX.
And that's good.
That's because we want to get the maximum the maximization of that specific class.
And next, we can use Normalize, which is a Keros vis utility as well, which basically enables us
to create the plots.
And that's about it.
So now we can actually just create a heatmap here.
This is from Cam.
This is how we have this since cool heatmap here, which generates I'll discuss the output shortly,
but they've been showing you and this is what this is.
The occurred here, this line that allows us to create those fancy heatmaps.
So let's run this and we should get that same output again.
Hopefully, we've done everything right.
There we go.
So it took about 10 seconds.
And that's inspectors, so when the CNN is identifying goldfish, we expect to look at the goldfish
and we can see it's looking mainly at the goldfish head because that's where it's the densest part of
this heat map.
Is this red zone here right above the head?
Similarly for beer.
What's interesting for the beer to me is that it didn't look at the beer fierce when making that decision
that this was a beer.
It looked at the texture, which could be misleading because I'm pretty sure that texture would come
up in other animals like dogs as well.
So CNN could be quite confused.
Sometimes with this texture arises in an image.
The assault rifle this one makes a bit more sense.
You can see this looking at more of the handle of the gun for the decision.
So this is quite cool.
So this is just grad cam.
What about Grad Cam Plus?
Well, Grad Cam Plus is a little bit better.
The researchers who who may have grabbed Cam Plus, made some improvements on the algorithm and to use
it, that's pretty much the same thing.
You just have you just import Grad Cam Plus from Grad Cam in this package here.
You just passed model and model modifier again to loss.
This creates a great object that will be using all the function functions, objects, by the way.
So sometimes they used interchangeably.
So then we have to loss the back end and then ultimately and then we normalize and then we create a
heatmap.
We use this in case I didn't mention before, and I don't think that seemed objet.
That's basically the function in MATLAB lab that allows us to get this color skewing here.
So that's and then we use that to get the image.
This is where we get the image out of it, and this is a color scaling nothing.
So let's run this.
Let's take a look at Cloud Cam Plus and Grab Cam Plus gives us a bit more information in a way you can
see.
No, that tells us, as CNN is looking more maybe similar to the last one, the bids.
Again, it's more of the best body, but overall it is looking at the best.
So but emphasis on this area sold rifle, similar area, but more of the gun this time.
So that's quite interesting.
Now we can take a look at school cam score comes in that another method that generates the class activation
map.
It's a gradient free method, unlike those the previous two we just looked at.
So this method could take a while, so it's wise not to run this on your CPU.
So I'll come back to this shortly.
Actually, let's run it now.
Let's see a it.
OK, there we go.
So it took a bit longer, actually.
And let's see, it's looking roughly at the same areas here, so we get the same answer.
So again, we can see this is quite useful for him.
Next, let's take a look at first cam, which, as the name suggests, it should read much faster than
score count because we can see score can was actually quite long.
So to implement fastest can, basically we still use the scorecard model here.
Let me see if that's what we looked at before.
Yep.
But to use go first to school again, all you have to do is know just said this parents to a value that's
not minus one because the minus one allows us to use original score.
And this allows us to visualize the class activation maps, which should be run much faster.
Yes, it's significantly faster.
It is about one tenth one.
Oh, sorry, one.
It's the speed, roughly.
So then you can see it gives us roughly the same areas again.
So this is quite good.
So that concludes this lesson on class activation maps using grad camp and school camp and first to
school camp and grad camp plus plus.
So we'll stop there for now, and we'll get back into the course where we'll start taking a look at
more complicated CNN's as we continue this course.
Thank you, and I'll see you soon.
But.