Hi and welcome back.
In this section, we'll take a look at filter activations for convolutional neural networks.
This might appear confusing to you guys.
However, I'll try to step through this section quite slowly so that you get a good understanding of
what these filter activations look like and how images are, how it's responsive, different images.
So let's get started.
So imagine that an input image like you have a random image that you're being at feeding into your CNN
now a trend of CNN, I should say.
So what do you expect to happen?
Well, what should happen in theory?
What you should know is that when you feed this image into the net network, certain filters are going
to be activated here because imagine this is this is a visualization of this filter here.
So imagine this filter finds this pattern would have a pattern of drawing here.
It's a random pattern.
It finds this pattern in the image.
That means that basically the filter turns on at that point.
So there are two things we can visualize here.
We can visualize the image basically the pixels in the image or areas of the image that corresponded
to that filter turning on.
Or we can actually visualize the actual activations as they go through.
So let's take a look at both.
So let's take a look at this image here.
Look at the image on the left and bottom left here.
What exactly are we looking at?
Well, if you look carefully, you probably wouldn't notice that this is the outline of a cat somewhat
to confuse two eyes right here.
And a bit of it is you can see this in this image.
It's the most clear, in my opinion.
So what is this?
Well, imagine we fed an image into a network and we wanted to see which, like which region of the
image matched that filter.
Whoop, Odom, explain this right?
So you get this.
But imagine each one of these are the images passed through each filter in our networks.
And remember, we have, you know, example network.
We had 22 filters.
These are the filter outputs for each one passing the input image.
And this on the right here.
As we progressed deeper into the network, you as we remember, the feature maps gets smaller and smaller
as we progress through the network because of the Max school is.
So that's why they're they're more blurry and contain less pixels.
So one thing you should notice and you can notice something quite quickly.
The outputs tend to be sparse and localized, meaning that the scattered like this and basically a little
bit region specific in some cases, and some of them didn't even turn on like this.
Five six right here didn't even turn on.
What does that mean?
I mean, sometimes we can use this method of visualizing filter activations to spot dead filters that
filters that don't turn on any input.
It can happen depending on the initialization of the way it's you might have fulfilled as a do nothing
and this add complexity to the network.
So let's take a look at something else here.
Here we have an image.
It's a --.
In case you didn't can tell, because it's quite small and you can see that filters are tuning on in.
Many different regions like the second filter here turns on, maybe for the blue sky in background.
This one turns on for this looks like a send the beach and some of the landscape.
This one turns on for the see, especially sequences learning.
A lot of the filters correspond to the many different parts of the image.
So in the previous section, what we were looking at this is the path I don't want you to be confused
about.
We were looking at the raw widths of the filters themselves, just looking at us, looking at the patterns.
Now we're seeing what parts of the image are turned on when we applied those feature detectors to them.
So here the filter out emissions, we looked to the output.
So again, here what we're doing, which is filter activations, is looking at the output generated
applied by applying a filter to an input image.
So let's take a look at these filter activations here.
This was the one I showed you in the previous slide the one on the left.
And you can see different.
These are the different layers.
So this block of filters and this block in this block are progressive CNN layers.
And you remember as we progressed through the CNN network to fit the MAX bill, the max pooling function
basically and even just applying a kind of filter reduces the output size.
That's why these look a lot more pixelated than this.
That's what we're visualizing here so we can see as we progress from the early layers left that are
deeply as in the right to feature map show less detail.
This shows that the image I mentioned get smaller, which is what I just mentioned as we progress through
to CNN.
In most cases, I mean, we can use padding and keep the resolution consistent rhetoric.
So this enables what this does, this is important now, this enables the upper layers to learn more
complex patterns because the upper layers here, the ones that are sparse in this case here correspond
to multiple different patterns of the input layers.
That's how this could seem confusing because it did confuse me the first time I learned this on my own
that the even though the feature maps are a lot smaller here.
These feature maps correspond to a number of different low level filters here, so they correspond to
much more complicated pattern detectors like this little dot right here might actually correspond to
something quite complicated, like a tree or palm tree or something.
Mainly because that mean mainly because it's a combination of the previous filters.
So hope you understood that.
Now, here's a simple example.
This is an example of the letter.
G looks like a nine.
I think it's a nine, actually, because the M this data sets, it's going to be the digit.
So this digit here has been inputs into basically a CNN.
And this is actually what we're going to do in our code.
Listen, we're going to see the activations for each filter has a capacity to see another.
Now the bright spots correspond to activations of dark spots correspond to new activations.
So you can see this a dead filter, at least not a dead films.
In the case where it doesn't respond to anything with the number 19 but respond to a one or two, who
knows, we have to experiment and see.
So you can actually see which parts of the image or which filter is responding to which parts of the
image.
So this is a pretty cool experiment to do.
So let's take a look at how we implemented this.
We just create a model that's what we did before the standard model that takes an input image and outputs.
But we have to stop the model and only output the feature lapse because that's what we want to visualize.
Then we look to image the normalized output.
That means between zero and one or whatever scale we want to use.
But zero one works well, but not what the visualizations.
And then we propagate over new input or any input you want to our new model.
And then we extract the feature map response.
Remember what we visualizing here is the outputs of the feature maps essentially that we want to use
MATLAB to visualize.
So we'll stop there and we'll move on to implementing the filter activations and the filter visualization
encoder using Keros afterwards.
What we're going to do is tickle taking a look at maximizing filters.
So that's going to be interesting.
So stay tuned for that lesson.
Thank you.