-: Hello and welcome back to the course on deep learning.
Today we're talking about Max Pooling
and we've got some very exciting slides coming up ahead
and even a special surprise at the very end of the tutorial.
So let's get started.
The first question is what is pooling and why do we need it?
Well, to answer that question
let's have a look at these images.
On these three images, we've got a cheetah.
In fact, it is the same exact cheetah.
On the first image, the image is positioned properly
and the cheetah is looking straight at you.
On the second image, it's a bit rotated,
and the third image it's a bit squashed.
And the thing here is that
we want the neural network to be able to
recognize the cheetah in every single one of these images.
In fact, this is just one cheetah.
What if we have lots of different cheetahs?
Here's a cheetah, here's a cheetah, here's another cheetah
here's a cheetah, here's a cheetah, and here's a cheetah.
And we want the neural network to recognize
all of these cheetahs as cheetahs.
And
how can it do that?
If they're all looking in different directions
they're all in different parts of the image, they're like,
their faces are positioned in different parts of the image.
Somebody's on the right hand side
somebody's in the left corner, somebody's in the middle.
They're all a bit different.
The texture's a little bit different.
The lighting is a bit different.
There's lots of little differences.
And so if the neural network looks for
exactly a certain feature, for instance,
a distinctive feature of the cheetah is
the tears that are on its face
going from the eyes or the
shadows that look like tears.
The texture, the pattern that is going from its eyes down
its on the sides of its nose that looks like tears.
That's a distinctive feature of the cheetah.
But if it's looking for that feature, which it learned from
certain cheetahs in an exact location or an exact shape
or form or texture, it'll never find these other cheetahs.
So we have to make sure that our neural network
has a property called "spacial invariance,"
meaning that it doesn't care where the
features are allocated.
Not, not so much as in which part of the image, because
we've kind of taken that into consideration
with our map, with our, with our convolution layer.
But it doesn't have to care if the features are a bit tilted
if the features are a bit different in texture,
if the features are a bit closer,
or if features are a bit further apart
relative to
relative to each other.
So if the feature itself is a bit distorted
our neural network has to have some level of flexibility
to be able to still find that feature.
And that is what pooling is all about.
So let's have a look at how pooling works.
Here's our feature map.
So we've already done our convolution
and we've completed that part.
And now we we're working with the convolution layer.
Now we're going to apply pooling.
So how does it work?
We're going to be applying max pooling.
There's several different types of pooling can apply.
There's mean pooling, max pooling, sum pooling.
And we'll comment on those towards the end of the tutorial.
But for now, we're just applying max pooling.
So we take a box of two by two pixels, like that.
And again, it doesn't have to be two by two.
You can choose any size of box.
And again, we'll comment on that
towards the end of the tutorial.
And you place it in the top left hand corner.
And you find the maximum value in that box,
and then you record only that value
and you disregard the other three.
So in your box you have four values.
You just disregard three, you only keep one the maximum
which is one in this case.
Then you move your box to the right by a stride
you select the stride once again.
So here we select a stride of two
and that's what you normally select.
You can select the stride of one, you can select,
so they're overlapping boxes.
You can select any kind of stride that you like
even three if you want.
But we're selecting a stride of two here,
and that's what is commonly used.
And then you repeat, repeat the process
you record the maximum.
Here, if you cross over, doesn't matter,
you just keep continue doing what you're doing.
So you still record the maximum here, zero.
Here the maximum is four.
Here the maximum is two.
Here are the maximum is one, zero, one, zero, two,
and then one.
So as you can see, a few things happened.
First of all,
we still were able to preserve the features, right?
The maximum numbers they represent
because we know how the convolution layer works,
we know that the maximum or the bit large numbers
in your feature map,
they represent where you actually found
the closest similarity to your feature.
But by then pooling these features,
we are first of all,
getting rid of 75% of the information.
that is not
the feature,
which is not the important things that we're looking out for
because we are disregarding three pixels out of four.
So we're only keeping 25%.
And then also because we are taking the maximum
of the pixels that we, or the values that we have
we are therefore accounting for any distortion.
So for instance, two images in which, for the example,
they cheetah's tears on the eyes are in one image
they're a bit to the left or a bit rotated to the left.
And in the other one they're a bit,
they're how they're supposed to be, or how we,
like, if we take one as the basis and then the other one,
they're a bit rotated to the left
the pooled feature will be exactly the same.
So you can see here,
if we are talking about the cheetah's tears,
then, let's say this is the four,
and this is where it was here,
then if it was a bit rotated,
so for instance the four ended up over here,
then when we are doing the pooling
we're still going to get the same pooled feature map.
And that kind of the principle behind it.
It's a very rough explanation, again, intuitive explanation
but that's the point of pooling
that we're still being able to preserve the features,
and moreover, account for their possible spatial,
or textural, or other kind of distortions.
And in addition to all of that, we are reducing the size.
So there's another benefit.
So we've got, we are preserving the features,
we're introducing spatial invariance,
we're reducing the size by 75%,
which is huge.
Which is really going to help us in terms of processing.
And moreover, another benefit of pooling
is we are reducing the number of parameters.
So we are reducing again, by 75%
we're reducing number of parameters that are going to
go into our final layers of the neural network,
and therefore we're preventing over fitting.
It is a very important benefit of pooling
that we are removing information.
And that is a good thing.
That is a good thing because
that way
our model won't be able to over fit onto that information
because, especially because
that information is not relevant.
Remember like at the very start we were talking about
even for human, us as humans
it's important to see exactly the features rather than
all this other noise that is coming into our eyes?
Well, same thing for neural networks.
By disregarding the unnecessary, non-important information
we're helping with preventing of over fitting.
So there we go.
That is what pooling is about.
And the question here is of course,
why max pooling, right?
There's lots of different types of pooling and you know,
why a stride of two, why a size of two by two pixels?
Lots of all these things.
And on that note, I'd like to introduce you to
this lovely research paper called
"Evaluation of Pooling Operations
in Convolutional Architectures for Object Recognition"
by Dominik Scherer from University of Bonn.
There's the link.
And the beauty about this paper is that
it's very, very simple, very straightforward.
So if you've never read a research paper before
which you'd like to give it a go
this is a great place to start.
It's very short, only ten pages, very easy to read.
And plus, the extra benefit is that
now that we've discussed convolution and pooling
you will be totally comfortable with everything
that they're talking about in this paper.
And you,
this is a great way to actually reinforce your knowledge.
So I highly recommend checking this paper out.
It'll take 20 minutes to read it
and you can even skip part two
which is called Related Work,
if it feels a bit far fetched or alienating
just don't read that part.
Just go straight to from part one to part three.
And the one thing that you do need to know about this paper,
they talk about a concept called sub-sampling.
Well, sub-sampling is basically average pooling.
So remember how here we were taking,
we were taking the maximum,
so in our square we were taking the maximum value.
There's a concept called mean pooling or sum pooling.
Sum pooling is you just sum these values up.
Average pooling or mean pooling,
you take the average value out of all of these.
And sub sampling is kind of like a generalization
of mean pooling.
It's, it's a more kind of generalized approach
to taking the average of, of these values.
And you can read a bit more about it in the paper
but otherwise just think of it as
average pooling when you're reading that paper.
And so that's where you can get some additional information
on this topic.
And now kind of let's recap, Where have we gotten to?
So there's our input image.
Then we applied the convolution operation
and we got the convolution layer.
And now to each of those feature maps that we get
we've applied the pooling layer.
So we've got,
we've done these two steps, convolution and pooling.
And now we're going to do something very fun,
something exciting.
We're going to experiment with this.
So this is a screenshot I took
from a tool created by Adam Harley from,
well back when he was at Ryerson University
of Computer Science.
And now he's at Carnegie Mellon I think doing his PhD.
And great tool.
So let's open up, let's have a look.
So you can find it.
You can't actually find it through Google.
You have to know the URL.
It's just hard to find it through Google
'cause there's no text here.
SC
Well just, this URL, (laughs) scs.ryerson.ca,
and then this stuff on the end.
And basically this is exactly what we are doing,
but visualize.
So here you need to draw a number.
So let's say I draw the number four.
And this tool will
put the number four here.
That's your image in our first step.
Then this is the convolution step, right?
And this is the pooling step.
And also pooling by the way is also called down sampling.
So pooling and down sampling are the same things.
So you can see it's applied convolution,
then its applied pooling
and you can see how it exactly works.
So you can see what kind of convolutions it has applied
or what kind of filters it applied, what they look like.
You can see what features it's looking out for.
And then it's applying pooling.
So it's reducing the size.
And you can see here that this is important, right?
So you can see,
that this is the convolved image,
and this is the pooled image.
And you can still see the same features.
It's just less information, but same features, right?
The features are preserved.
That's the important part.
And moreover, if you know, if all four was a bit
to the kind of like rotated a bit to the side.
it would still be able to pick up
very similar pooled layers.
And then after that it's got more layers.
We haven't talked about that yet.
So then it's got another convolution,
convolutional layer here which we actually won't have.
And then it has another pooled layer,
but it's basically just repeating that same process.
And then after that,
this is what we're going to be talking
further down in the course.
It's got the fully connected layers, and so on.
But you can definitely play around with that.
So, if I delete that you like, if I draw a seven,
you'll see that it actually tells you the guess,
is it guesses that this is a seven.
And the second guess, the second likelihood, is a three.
So you can draw it some challenging things
and see if it can pick them up.
So let's say if I draw something that looks like a zero
but it's not a finished zero, will it pick it up?
Now this, this time it didn't pick it up.
Looks like a nine to the, to the image.
What if I kind of like finish it like that?
See now it thinks it's a zero or a nine.
And you can see over there
what's lighting up, the zero or the nine.
But we'll talk about that part for the down.
Let's do one more.
Let's say
like eight,
I think eights are pretty hard for this.
No, picked up an eight.
So you can see that goes into an eight
and then like after that it stops being recognizable.
This stops making sense to us humans, right?
These features that it's working with.
But at the same time
it is correctly recognizing that it's an eight.
Yeah. So definitely play around with that.
You can draw a smiley face, see what happens then.
Looks like a three to this
to this tool because the tool is obviously trained up,
only on digits from zero to nine.
So it has to recognize something there
out of those, and it recognizes a three.
It's like in life when you see something like a
a type of fruit that you've never seen before
like a
custard apple or something
and you think that it's, like it's a,
it's a pear because you've never actually seen one before
you don't know what to classify it as.
Same thing here.
So it hasn't actually trained on smiley faces
and that's why thinks it's a tree, it's the three.
So there you go.
It's a very powerful, powerful tool.
It'll be helpful for you to play around with it.
Actually you, when you put your mouse over a pixel,
it shows you where the feature detector was
to pick up that pixel
so you can see where those, these pixels coming from.
And also, so you can see how the filter was kind of like
going through the image exactly how we talked about
in the course.
And here you can see, you can see the pooling.
You can see that the pooling is done with,
the pooling is done with a
little square size of two by two.
And you can see that it's a stride of two, as well
just as we discussed in today's tutorial.
So there you go, play, have a play around with that.
And I hope you enjoyed today's session.
I look forward to seeing you next time.
And until then, enjoy deep learning.