Hi and welcome to our summary chapter of CNN's This one should help you put it all together so that
you can come away from these lessons having a very good understanding of what convolutional neural networks
do, how they're trained and how we get how they work in classifying images.
So let's get started.
So firstly, let's do a recap.
Remember what convolutional layers were?
They were the filters or kernels that we basically applied to the image the sliding window here to get
the feature maps output.
So it accomplished.
No operations occur when we can evolve or filters with the input image, slathering it over the image
which you've seen destroyed, and padding, which we'll go over shortly.
And it produces two feature maps of what's here.
That's what we get.
And remember, this is a there's a formula which you will see in the next slide that tells you how we
get the size, how we determine the size of the feature map that we get.
So remember, the size depends a lot on stride and padding as well.
So we have this whole formula here.
So you remember that we just patted zeros around the image initially that allows us to maintain the
size so we don't actually lose image size if we have padding equal to one.
And by having a stride, remember when we increase the stride, we decrease the feature map size.
So by keeping us straight at one that generally will bend T and the size of the image, assuming that
the padding is one as well.
Next, we looked at really, really was the activation function that basically allows the neural network,
or CNN, to introduce nonlinearity to its learning capacity and the real activation function.
Basically to tell Tullis that when we when we apply it to a feature map, all the values are less than
zero.
The negative values are clumped to zero and all the values that are positive over zero are left alone.
So that's how we end up with this rectified feature map here.
Next, we looked at max pooling.
Remember, the max pooling operation is a way we can actually downsample or reduce the feature map size
and still retain valuable information.
It doesn't lose much in this process, and it reduces the number of parameters in our CNN, which is
a good benefit because the more parameters we have slower is the trend, but the more tendency it is
over it.
It just becomes a more difficult problem at that point.
So by having this downsampling max balloon operations, we can improve our training process much better.
And you can see with the max pooling Sofia, we just pick the largest value the square in the square,
in the square, in the square.
And then we put it here.
And again, I still have the same error here.
This to have to choose the largest values.
This one, six or seven should be 253.
Apologies for that.
Next, we took a look at the fully connected layer.
The fully connected layer basically just flattens all of the feature map outputs from the max pool operation
into one vector, one a one dimensional vector, one long vector.
You can think about it where every node is connected to another node.
So this 123 is a new ID here.
And of course, it connects to this node, also to this node and vice versa.
So if you have a lot of nodes here, it's a lot of connections going on.
And then finally, we took a look at the SoftBank's layer, which is the end of the year to CNN.
And basically, this converts our scores to probabilities.
And by using this formula here.
So that's this is a that's a full CNN put together.
We have an input image, we have a conveyors, we have a max blue operations, we have to flatten function
and then we have which is connected to the fully connected layer here.
And then we have the output nodes, the output nodes of basically the final nodes, where each output
corresponds to a class score.
And you can see that in the in the diagrams, we have fleet of one.
This is another way to represent the CNN, which is actually prefer this way because you can actually
see and visualize the filter sizes here.
You can see the treaty size of the feature maps as they're produced.
And it looks nice as well as pretty.
It's pretty cool.
And these are the calculations to determine the number of parameters in each of these layers.
And they are non-tradable parameters here, which involve the max flattening real disorders functions
and essentially the fully connected or densely.
It's often called densely.
And some libraries and literature that gives us that's where the bulk of the parameters are in this
case, as well as in the CNN two.
And then we have the final output of Celia here with these parameters as well and summarized in this
table right here.
So this is the training process.
Let's let's take a look at this.
So we have a batch of images here.
These images are fed into the network.
Now we can feed it.
We do feed them one by one.
However, we can also feed them in as a batch like that and the batch we get by feeding into batch.
We get these outputs here.
Now we remember we use these output scores to calculate a loss that's using the cross entropy loss function.
And now we have that quantified loss.
We then used that loss to apply by propagation to update the gradients.
So let's take a look at the overview of the training process.
So remember I said we just we have the loss of function values from the forward propagated batch of
images and then we use that propagation to back propagate and update the weights or gradients as it
moves from right to left, going back into the network and then the process by which we apply by propagation.
This gradient descent gradient descent effect if it basically encompasses propagation, but it basically
speaks about algorithms that optimizes with how we update IC, how much we update the gradients by,
and that's the lambda parameter, which we saw.
So this overview, we took a look at how a standard model is designed and defined.
How do we see initialized with random values or batches of images, typically eight 256, depending
on whatever can fit into a GPU or a system RAM and then forward propagated should have seen in model.
Then we use back propagation with typically mini-Budget in descent or something like Adam, or might
have one of those optimized algorithms and we did gradients right to left.
And then we basically use the updates to produce tweets.
The updates produced tweets that have a lower loss.
And this is done for the entire dataset.
So we keep for for what?
Propagating these images continuously into the network until we've completed the entire dataset.
And that's called one epoch and we typically train for five to 50.
You can train for more, you can train 500.
It's basically diminishing returns at some point because eventually the lops loss stops decreasing.
So you tend to use 50 bucks as a good ballpark figure.
Sometimes you can get away with five, depending if the dataset is quite simple and the network as well
optimize.
So let's go over what quickly?
What batches many batches of iterations of epochs are because these terms confuse a lot of beginners.
So let me, we can feed them just one at a time.
But as we saw previously using batches as a low, so much more faster training process.
But what a mini batches.
Well, the mini batches effectively matching itself.
It isn't.
It isn't anything different.
It's just what we call the process by which we batch data and feed it into the property before it propagated
into the neural net that so many batch.
Do you know, iteration?
What is an iteration?
Well, an iteration is a number of mini batches that to take the data taken to complete one epoch and
one epoch is basically where we finish passing the entire data set to the network.
So what are some advantages of CNN's just to recap?
There's scale invariant.
Remember how well Max Boulia basically kind of ensures that we don't lose that and variance as we don't
sell both of the filters then allow for parameter sharing because of the highly correlated pixels in
an image.
This allows for parameter sharing with weights, as well as also because we're also using the same filters
for all parts of the image.
That basically enables parameter sharing as well.
In the column for filters, which is a very good thing, reduces the size of the number of parameters.
We need to learn from the images and then the positive connections basically expands on that point.
That's also because of the highly correlated pixels and because we're also sharing parameters.
We do.
We need a lot less connections.
So overall, it's a good thing.
And you remember these are the assumptions.
CNN stick low level features that there was a features that are grouped close together with a local.
So that's what Lucho means.
Global features are basically simple.
Features features a translation invariant, which means it can be anywhere on the image, and high level
features are made up of low level features combinations of them.
So hopefully that recap of CNN.
So hopefully it solidifies everything we've done so far with.
We'll stop there for now, and what we'll do will now take a look.
Basically the history history lesson of deep learning and AI.
Thank you.