In the previous lessons,
you saw the impacts that convolutions and
pooling had on your networks efficiency and learning,
but a lot of that was theoretical in nature.
So I thought it'd be interesting to hack some code
together to show how a convolution actually works.
We'll also create a little pooling algorithm,
so you can visualize its impact.
There's a notebook that you can play with too,
and I'll step through that here.
Here's the notebook for playing with convolutions.
So first, we'll set up our inputs and in particular,
import the misc library from SciPy.
Now, this is a nice shortcut for us because
misc.ascent returns a nice image that we can play with,
and we don't have to worry about managing our own.
Matplotlib contains the code for drawing
an image and it will render it
right in the browser with Colab.
Here, we can see the ascent image from SciPy.
Next up, we'll take a copy of the image,
and we'll add it with our homemade convolutions,
and we'll create variables to keep track
of the x and y dimensions of the image.
So we can see here that it's a 512 by 512 image.
So now, let's create a convolution
as a three by three array.
We'll load it with values that are pretty good
for detecting sharp edges first.
Here's where we'll create the convolution.
We iterate over the image,
leaving a one pixel margin.
You'll see that the loop starts at one and not zero,
and it ends at size x minus one and size y minus one.
In the loop, it will then calculate
the convolution value by
looking at the pixel and its neighbors,
and then by multiplying them out by
the values determined by the filter,
before finally summing it all up.
Let's run it. It takes just a few seconds,
so when it's done, let's draw the results.
We can see that only certain features
made it through the filter.
I've provided a couple more filters, so let's try them.
This first one is really great
at spotting vertical lines.
So when I run it, and plot the results,
we can see that the vertical lines
in the image made it through.
It's really cool because
they're not just straight up and down,
they are vertical in perspective
within the perspective of the image itself.
Similarly, this filter works well for horizontal lines.
So when I run it,
and then plot the results,
we can see that a lot of
the horizontal lines made it through.
Now, let's take a look at pooling,
and in this case, Max pooling,
which takes pixels in chunks of
four and only passes through the biggest value.
I run the code and then render the output.
We can see that the features of the image are maintained,
but look closely at the axes,
and we can see that the size has been
halved from the 500's to the 250's.
For fun, we can try the other filter,
run it, and then compare
the convolution with its pooled version.
Again, we can see that
the features have not just been maintained,
they may have also been emphasized a bit.
So that's how convolutions work.
Under the hood, TensorFlow is trying different filters on
your image and learning which
ones work when looking at the training data.
As a result, when it works,
you'll have greatly reduced
information passing through the network,
but because it isolates and identifies features,
you can also get increased accuracy.
Have a play with the filters in this workbook and
see if you can come up with
some interesting effects of your own.