Hi, welcome back.
So in this lesson, lesson nine, which you will see here highlighted is we will take a look at filter
visualization and filter activation visualization.
So open that notebook, which I already have open here and let's get started.
So firstly, in this lesson, we're only going to use cameras to visualize to do these visualizations
simply because it's much easier than using PI two.
It's not that PI is that much more difficult, however, because this lesson is more of a theoretical
lesson to teach you how CNN has learned.
It's not really something you would do much, and in practice, it's more of a way to reinforce what
you've learnt.
I will probably just stick with carrots because it is easier to easier to implement, and there are
some cool Keros visualization libraries that allow us to to do some visualizations afterward, which
you will see in the next code section section two, not Section nine, which is this one here.
So let's see.
Let's do this.
Let's create that saw this.
Listen to this.
So to do this, firstly, we're going to have to train a basic CNN on the amnesty to set that someone
will be.
That's the one who's filters will be visualizing.
So let's train that.
So you've seen all of this before.
We're just basically going to load of data sets here, run all the inspections, which we don't need
to do, but just it's good practice here.
We were going to reshape the data and due to pre processing and whatnot and coding of the data.
So that's all done there.
Next, we're going to define our model.
So we defined the model architecture here and now we're going to train.
So that's all trained for five 25 epochs.
Let's do.
The 10 will still get maybe close to 90 percent accuracy, I believe, but we'll find out shortly.
So let's wait for this to train.
OK, so we're finished reading and we got 90.
We got 92 percent point ninety two point three percent.
It's not too bad.
Attorney box that was pretty quick to trim.
So now let's start visualizing the filters.
So firstly, let's take a look at let's look at earlier names here so we can do something where if we
do it, layer top name in this list comprehension here.
So we go through the model is we're able to access to lawyers like this here.
So let's do that and you can get it to get the output here.
Each one corresponds to this point to this object here at the memory address.
Don't worry about what those numbers mean.
They're not too scary.
But importantly, this isn't what I wanted to show you.
This dictionary has two layers in that CNN.
So get two ships.
Two, Let's get the ships of our kind of filters here.
So to do that, what we do, we just loop through the model is here.
And if it's a conveyor, we just continue here and we just print.
We get we used leered dot, get weights to get the weights and the filters and biases and into the weeds
and biases, and we just print delirium here as well.
So in this particular, particularly if you're going to be using this here.
This is probably some leftover kodaira by mistake, but just it's good to show you that we this is how
we could actually get these things out of here if you wanted to get them in a usable way because this
is going to keep overwriting as we go to the network.
You'd probably want to store these in a list or the dictionary as well if you wanted, but a list is
probably not practical.
So but anyway, the point of this block of code is to show you the icon, filter size and the number
of filters here.
So you can see for the first layer, we have the tree by tree by one 32 filters.
The next one, we have tributary with filters.
We have 32 feature maps and 64 as well, 64, which is for this, for this one.
So that's basically how we get the ship to the network.
Let's run this week's show at work.
So.
So let's take a look at the widths of our first conflict.
So using the did not get which function which you've seen before, we just access the first live network
by pointing it to zero.
We just get the weights here.
So remember we can get the weight of weight or filters and biases here.
So let's take a look at what that actually looks like.
So remember, we're particular shape here.
So the shape you'll see, it's where we're looking at tree by tree filters and we have 32 of those filters.
So let's take a look at this.
So what would miss?
What's what's what's this messy output here?
Well, basically, I'll tell you, well, this output is basically the filters.
Here we have this.
This is a consistent, multidimensional array has why there's so many brackets and these are the filter
widths for the 32 filters that we have.
So it's tree by tree.
And then we have 32 weights here.
So that's just the output of that's just to help you understand what you're looking at.
And then the biases you remember, the biases are shared for the filters.
So if we have 32 filters, we'll have 32 biases in that case.
So we have all the biases here.
So now, in order to visualize these filters, we have to normalize them or skill them between zero
and one so that we can use MATLAB to properly plot them.
So to do that, it's actually quite simple.
But some Nampai functions here, so we just use filters dodman filters of Max to get the minimum and
maximum value off or filter widths.
And then we just skill them.
Using this formula here to this ends up skewing all the values.
So we just do some printing here just to show you what's going on.
So you can see the min was minus point one four and Max was 0.3 six.
At least that was when I ran it before.
It might be different this time, and it is a little that's roughly the same.
And anyway, after normalization, you can see we've now pushed this mean to zero and Broaddus Max all
the way up to one.
This will, I mean, this isn't absolutely necessary.
However, it will produce better plots, better visualizations.
So no, we can move on to visualizing our filters.
So let's run this block of code and I'll run through it quickly.
So what's happening here?
Well, what we're doing here, we're going to plot the first few filters and set what we basically said
the process initially.
That's what a comment thing.
And these are the filters we're putting and this is this IEX as well.
So I access index that we loop through as well afterward.
So for Renge and the filters and filters with a number of filters here 22.
So we just get the filters here.
They're extracting each year, which are all the values for that specific filter in the loop here.
I could also use that I am not sure why I used AI separately.
Maybe one way this one starts and.
Zero, this one starts at one.
That's why he says this for the subplot, plotting apologies for the confusion is just sometimes it
happens to me quite a bit where I revisit code that I've done a month or even two days later sometimes,
and I kind of forget what I did.
So bear with me sometimes and apologies for that in advance if it happens again.
So we just use the supplied function here.
So the software function for 22 subplots, by its subplots by four, and then we have that index here
so horrendous and will give us that output, which we just saw below.
This was also the output I used in the slides, which you read about, which I hope you remember seeing.
So what we're looking at here is the visualization of the two bits of the tree by tree filter.
You can see one can clearly see three different pixels.
So more apparent, maybe here.
I'm not sure what it is, but it's roughly the same anyway.
So you can see we have nine different tiles or squares or pixels actually in this filter, so you can
see what these are, what the filters are looking for in the image.
So this one, it looks a bit random.
However, it is learning important things that correspond to different parts of digits.
So that's what we're we're seeing here.
So now let's take a look at filter activations.
So if you remember correctly, filter activations are basically the feature map output we're looking
at.
So remember here we were looking at the filters themselves.
But remember, after the first conv of the Liberty twenty eight by twenty eight input image?
Well, that gives a feature map that is 26 by 26, and we have 32 of those feature maps to look at.
So to do that, let's take a look at how we do filter activations here.
Now, this can seem a bit confusing.
And believe me, some of these filter visualization techniques actually do confuse a lot of researchers
as well.
So don't feel bad if you're confused what we're doing here.
Firstly, we're extracting the outputs of the top seven layers here, so that's what we're doing in
the first step here.
Next, actually, the first two layers, I should say, not the first seven.
Sorry about that.
So next what we do here, we create a model that will return the outputs, given the model input.
So that's what we used a model function from Carousel and we have the model input here.
We model outputs, which are the outputs of these first two layers, and that's basically our activation
model.
So let's run that and let's take a look at the outputs.
So it's as I said, we got the first two top layers, which are the first to convey is great.
So now what we can do, we can take a random image here.
So we're taking the twenty second technically 23rd image of our test dataset.
We were shipping it.
We just changed the image size and we're displaying it.
So this is just to illustrate what the input image is.
OK.
Next, what we're going to do, we're going to get the output after the second filter.
That's after the redo activation.
So to do that, we take the activation model that we created here.
We send this image tensor.
That's this image here that we've reshaped into this correct form.
I notice a lot of the little steps, but don't worry, at least all the code is here, so you can learn
from watching this code if you want to spend some more time on it.
So anyway, the activation model gives us the activations here, and we can see we can see how many
activations we've extracted.
So of basically this tells us how many activations we got.
We got two sets of activations so we can take a look at what the first one.
Remember the activations for this number?
We're getting it for each layer.
So the activations for this here can take a look at the first one is going to correspond to the output
of the first convolutional layer.
Remember, I set it up and is twenty six by twenty six and we have 32 filters.
That's exactly what we expect to get.
So what we're effectively doing here, we're just decomposing to CNN and its different layers and extracting
the outputs midway as we forward propagate image truth.
Next, we can look at a second because remember, we got two activation two types of activations types
I should say the two groups or containers of activations, and each container contains this and that
this is the second activation layer now.
So let's take a look at it here.
And just to recap, this is a good v knowledge.
This is a model summary here.
So what we what we've done, we basically extracted to filter activations freedom about feature map
activations from the first layer here.
That's why the output shape is that and this is a second one, and that's why the output shape is that.
So hopefully.
You're still with me.
And then what do we do now?
Let's take a look at the Fort Collins filter.
Actually, I should say the fort to pick him up there should be a little bit better.
And so we do that in the extreme because remember, it's sort of index zero one two.
So the fourth one would be the tree.
And basically, that's we just plotted.
We plot everything had this one.
This value isn't used.
So we just use this one and we use the various color mapping.
We can use a grey or one of the many of the others, but they're just as nice for this illustration.
And there there we go.
So what are we looking at?
Well, this basically is the future map to the Fort Feature map in this layer.
The first activation Cornelia, this see, it's 26 by 26.
That's what that's what we're visualizing.
This is the output of its of that feature map from that filter.
So that's pretty cool.
So we can take that same this little bit of code here that we've used and create a little loop or function
that loops through all of the future maps and plots in a subplot or even a subplot grid, I should say.
So let's run that function and run the code and we'll get a nice feature.
Map activations, hopefully soon.
Yeah, there we go.
So you can see these are the feature maps from the first conflict.
The outputs of them, that's when we input an image.
Six.
You can try different inputs to see what's what's going on.
So you can see what the brighter with a more the brightness of these colors you see here.
Like this one?
This one is the most activated or perhaps one of these tree here or this one on top here.
Two filters that correspond that activated the most upon seeing the number six.
So hopefully this lesson was useful.
I know it's a bit confusing.
I don't want you to get too confused right now because this type of stuff isn't isn't a focus of the
course.
The focus of these lessons is to allow you to visualize what the CNN's are doing internally.
This is also good for research students who want to have a deeper understanding of what their model
is actually doing or what their filters are doing, so I left it in for that purpose.
And but however, as a practitioner, you may not need to do this as as much unless you are trying to
troubleshoot an issue.
So we'll stop there for now.
And then the next section, we'll take a look at filter and class maximization.
So I'll see you in that section.
Thank you.