Hi and welcome back.
We're about to start the lesson on activation layers, and we'll talk a bit about religion, which is
the most important useful activation layer for science.
So let's take a look and see why activation is important.
So it's important you think about the purpose of an activation function.
This could confuse a lot of you, so be slow and try to introduce this topic gently.
Firstly, remember, we're trying to learn complex patterns in our data.
These complex patterns mean that the data is so vast and has so much different variations that small
changes like imagine you're watching the imagining trying to identify digits and the difference between
a one and a seven isn't that much is just this one has a bit.
It's a bit more straight and vertical as a seven can have a slight angle and it has a top horizontal
bar went up to seven.
But those are minor differences.
If you think about it and we do different the way our mind actually knows that it's a seven and a one
is that there's a decision boundary in our brains, and our brains basically are just a bunch of neurons.
That's effectively what it is that fire and activate and be able to identify what's what quite efficiently
and quite quickly.
It's actually amazing how efficient our brains all this.
So this gets to the point where we talk about non linearity.
So remember, I talked about that decision boundary between a one and a seven.
Now imagine we have a whole host of digits.
You have all the numbers and alphabet all the digits between zero and 10.
That's a lot of different classes, so you need to have a complicated way.
Basically, it's a nonlinear type function that can take these inputs and produced output that we want.
Well, that's the upper that's correct, I should say.
So that's for the non linearity.
It's it's a it's a basically decision boundaries that vary and depend on many different factors.
It's not a linear type mapping function.
So that's hopefully that gives you some intuition on why activation functions are important and what
they allow CNN's to do.
So let's take a look at some simple activation functions.
So we're going to we're going to talk a bit about really in depth and we lose advantages in CNN training
because it's actually quite simple computation and it's faster train, which means in fact, it's fast
to train and it doesn't saturate.
So that's also a good point.
So let's take a look at some simple activation functions, and you're seeing this function here, this
function here, this function here, which is really to rectify the linear unit.
It's what it stands for.
So what does this mean?
Let's let's take a look at the redo in this case.
This means that all of the values, all of the inputs into function are less than zero will always be
zero.
So that's what a this mathematical function is seeing here.
If Z is less than zero at zero.
However, if Z is greater than zero, then it takes the value of set.
So that's the nonlinear linearity part of this function right here.
So as you can see, this is another way to represent reload.
Basically, it's a max of everything that's over zero and the max, meaning that it just gets the value
of X here and everything below zero is zero, basically.
So we leave all positive values alone and we change all negative values to zero.
That's really in a nutshell.
So let's take a look and see how we apply a real mathematically.
Let's see what it does to our ultimate future map.
So this is the output feature move we get by involving this input image with this filter or kernel,
and you can see we have two one minus one minus one, one three to one and minus five.
Remember what really we said?
We change all the negative values to zero and leave all the positive values alone.
Well, that's simply what we do.
All the negative values here becomes zero.
You can see them here, and all the positive values remain the same.
That's it's quite simple.
That's what really does.
So real is often considered to be in the same layer as a conversion layer.
We just specify what activation function we're using.
However, a lot of the literature you will read, some authors or researchers like to call a renewal
a separate layer when in fact some people are lumped together.
I'm of the opinion that can be bought.
However, you want to infer, just make sure you're consistent when you're seeing it, as this neural
net has nine layers and you want to compare it to another one, just make sure you compare them correctly.
So this is an example of how the re-look operation works after we take this feature up and we apply
it.
Sorry.
So you can see the output here.
You can see what's happening is that all of the dog values, which are the negative values are being.
Basically eliminated in a way, and all the white values are roughly staying.
So that's it for the real, real lesson.
Next, we'll take a look at the pooling layer of CNN's.
So stay tuned for that lesson.
Thank you.