Hi, thank you, and welcome back to the course in this section, we'll take a look at Raisinets.
The resonant network is actually one of the best CNN architectures you can use because they solve a
number of problems which will explain why, and we'll actually go to the mathematics in the next section
to show you how residents actually solve this problem.
So let's talk about firstly, classical CNN's they do look like this.
Remember, we have multiple layers and max pooling, then a fully connected layer at the end, fairly
basic.
You should be fairly familiar with this by now.
It's just linear sequential sequence of linear operations.
So what's the problem?
What problems can happen?
Well, in reality, with Classic, with the classical student architecture, a lot of problems arise
during training.
Often performance gets worse when we have when we increase the number of layers because his is an illustration
of that concept.
This is the expectation you think what's going to happen when you keep training?
However, in reality, this is what happens.
Your performance can get worse over time.
That's when it starts over fitting and the generalization basically it stops being it stops the ability
of the network to generalize, which isn't a good thing, obviously.
So the reason for that is a number of reasons for that, too.
But the main reason for that in the deep CNN's is a problem called exploding and vanishing gradients.
There are two different problems I have for here, but there are actually two different problems, and
I'll show you how they arise in deep networks with number of layers and layers we have and derivatives
that must be multiplied together to perform all gradient updates.
So imagine, instead of like having five multiplications, we now have like 19 or 20 or 50 even more.
And you can see at that point, if a gradient is large, it's just going to explode.
It's going to affect all of the downstream gradients as well.
Likewise, if the gradients of small, it's going to decrease exponentially or vanish, and that's a
problem that resonance actually solve.
So let's take a look at this.
His or the resonant, his whole resonance solve the problem.
This is an illustration of the standard sequential linear operations of a basic CNN.
We have to convert real a hill, followed by another convert regular here.
So what does Reznor do differently?
Well, it connects to input that was going into the first layer to the second conflict.
So now the second conclave has two inputs.
It has two input that was initially meant for the previous layer, as well as the output of that conflict.
So how does that?
How does it how does a short circuit solve the problem?
Well, we'll take a look it up in the next section properly, but I'll explain to you in a high level
note.
What happens here is that if the value, the output value that we had some biases got too small here,
basically it would disappear.
However, because we have the input here going into the second layer here, it doesn't actually vanish
anymore.
It remains a decent value that can be calculated.
That solves a very big problem in deep learning.
With two players, vanishing gradients basically kill our results in the end.
So this is how the resonant model is designed.
They've got tons of these layers, which allows us, which allows the CNN to learn very, very deep
features and allows it to generalize quite well to.
So this is allows us to get to this level, the expectation we want.
So as opposed to this one.
So just to recap about resonance, because I didn't mention this in the beginning, but resonance were
introduced by Microsoft's researchers in 2015 and in a paper titled Deep Residual Learning for Image
Recognition from he and his group of researchers, which I don't name here.
That's what it all means.
Subgroup of them.
And you can see the comparisons of the network.
They've got a 2D folio.
They introduced a 2D folio resonant.
And actually, to be fair, that's one of the basic resonance resonant, resonant 52.
Resonant one to one resonance, 150 to something like that, I believe.
So this you can see drastically how much more layers we're able to pack in a resonant.
And this allows us to end still in many, many more features, many more patterns that helps performance
so much.
So in the next section, we'll take a look at a little bit of the mathematics behind how residents work.
It's not that it's not that complicated, so I'll try to break it down for you in a very simple way.
So I'll see you in the next section.
Thank you.