Hi and welcome back.
In this section, we'll take a look at some of the underlying maps that explains how residents work
and how they actually solve the vanishing gradient problem.
So it's quite simple.
So don't be scared.
The maps in this slide is actually quite simple.
Let's step through it.
So remember correctly is combined with the real operation is effectively just a simple, linear operation.
That's what this first line says here.
It says the output.
That's Z L one l plus one is equal to the weights times the L input here.
Plus Tobias Ayo's input W one is the weights of this layer here, the first layer plus B one, which
is the bias of the first conclave.
So that's before the rhythm operation and we have the added value around it here, which is the Z one
C L1, which is the output of this node.
Next, we have Z two zeal to, I should say, which is the output of this node, which is this operation
here takes the output of this here three times, by the way, we'd say plus, Tobias, of this conflict
here.
So again, that's a simple operation and then whipped out the short circuit, which re lu.
This is what it looks like here.
The output, this is just continuing here.
So it's one two, three four.
And then this is the output here.
What the short circuit now pay attention to this here.
You can see we have g.
This is where the real activation function of Z L2, which is the output here, plus a one initial input.
So what does that mean?
It's actually quite simple.
This is what it means here.
This is when we expand out what the what L1 is and here so you can actually see the expansion.
However, I wouldn't focus too much on the details of with that.
I want to focus on this one here.
Now what happens if W or oh my host, we saw the bias is very small near zero.
Take a look at this equation here.
That's the output of the Corsican gondolier IL +2.
So you can see if this is close to zero.
This is close to zero e l plus two is equal to ends up being close to equal to G of F.
This is a good thing, however, because if if it wasn't for this E input here, this would have gone
to zero.
Do you see that?
So this is what this line says.
This is the important part of the slide.
So you can see by having the input here, it prevents the output of this node from being zero.
This thus allows us to solve the vanishing gradient problem quite effectively.
So now you can see that solving that vanishing gradient problem allows Krasnov networks like the rest
in a telephone less than 50, and as I said, his arrest resonant 150.
It is actually a resonant tool, one you can see.
It allows them to be very, very deep, better and that we're very big and we don't need as many fully
connected layers and the and here we can just have some simple, fully connected layers or none at all,
actually sometimes.
And it seems a lot of parameters right there.
So vigyan, sorry, the reason it has been a very, very good network hurts.
It has achieved remarkable performance and is actually a people, a 20 or 21 people that says residents
are all you need.
Basically, what are people established was that even though a vision transforms vision transform,
that networks have basically surpassed state of the art and resonance and resonance networks and seen
an image that performance categories.
The researchers showed that were just some minor tweaks.
Residents are able to achieve similar and even better results.
So I would suggest you guys, if you need to use a network radio as a network or pre-trained network,
which will all be doing in little lessons and colab.
So don't worry, we'll actually start using these networks.
Get some hands on experience with loading these networks, loading pre-trained versions of these networks,
doing transfer learning, fine tuning.
So it's going to be exciting time when you start playing with these advanced CNN's.
So I'll stop there for now, and we'll take a look in the next chapter at a different type of CNN mobile
unit, which is basically not trying to achieve a remarkable performance.
It's trying to achieve achieve remarkable efficiency.
So I'll see you in that section.
Thank you.