Hi and welcome back.
We're about to take a look at denseness, which basically effectively resonates, but better.
So let's get started.
So dense that was introduced in 2016 by a joint effort by the Cornwall University, Shinhwa University
and Facebook's EIA research team.
And in fact, denseness won best people in 2017 CPR Conference, which is the Premier Computer Vision
Conference for publishing.
So that's a big deal.
And it was able to obtain an even higher accuracy than resonate with fewer parameters.
This is just a preview of the architecture we'll we'll be discussing shortly.
So how did it do that?
Well, basically, it solves the same problem that residents of which is a vanishing gradient problem.
And as we know, just to do a recap when networks get very, very big, having a very small gradient
in that network can basically cause the gradients along that chin to become zero, which is not good.
It basically means that you're kind of wasting parameters, wasting computation time at that point.
So then, since we're able to solve this concept by introducing something called collective knowledge,
where each layer receives information from two previous layers.
So let's take a look and see how that's implemented.
So this here is all classical scene in architecture.
You remember this?
This was the feature maps, feature map outputs and blah blah blah.
We have okunnu layers to reload and all of that stuff.
So you should be familiar with this.
Now, this is a resonant architecture resonance basically introduced at Short-Circuit here, so you
don't always get vanishing gradients.
So what did dense density do differently?
Well, let's take a look.
Dense that's connect to have multiple connections so you can see.
Let's follow the path of this read.
One can see it connects here just like a resonant short-circuit.
Then there's a yellow pad that goes all the way to here, and then there's a green one that goes all
the way to here and a pink one that goes all the way to here.
So you can see this input is received at all the intermediate CNN layers.
So each layer receives information from the previous layers.
And the growth rate, which is the additional number of channels for each layer, is controlled by a
parameter called K.
So that's not all there is to then.
That's the basic dense and composition layer has this foundation here.
They have a pre activation batch norm.
And then the second layer with Crabtree filters, that's basically it.
However, they also introduce something called bottleneck layer.
So what are these bottleneck layers?
Well, remember we have batch known Mary-Lou, and then it's followed by a one by one layer here.
And then that's a bottleneck point right there.
And then we can have a batch known and a tree by tree columns here.
So you can see this sort of like a squeeze or the file module similar to the file modulo, squeezing
it sort of a squeeze and a squeeze expand layer.
It's actually quite similar.
So this was the architecture diagram used to buddy that Transnet researchers and the people, and you
could kind of get a feeling of what it is here.
It's not that descriptive, but basically, here's a note I put in.
There's one by one converters, followed by a two by two averaging pool.
And these are used as transition layers between two contiguous, contiguous dense blocks.
So you can see they have a block here block here that is followed by a two by two average pooling.
And that's the transition layers.
So this here then Senate introduced several networks of different depths this that 121, 169 to one
and 264.
You can see that it's quite deep.
And growth rate is controlled by a but I mentioned for you guys.
So let's take a look at performance now.
You can see the error rate for theinternet was quite good and you can see it going down as network layers
increase.
Similar for the top five or it, you can see it went down to quite low values here.
But let's compare this to Reznik now, given the number of parameters given less parameters here.
You can see denseness performed quite similarly, basically better with less parameters.
Even like look at this one, denseness 121 is getting an error rate of twenty five point five, whereas
resonated for IS getting error rate of twenty six point five.
Accuracy reversed, sensitive using accuracy, we use error rate.
I do tend to prefer to use accuracy, but some researchers keep this standard going forward, so let's
just stay with it anyway.
So basically, you can see the lowest error rate here for theInternet 264.
It still performs better than a resonant, which has twice the amount of parameters.
Similarly, for flops, which is the computational inference number of calculations that it does in
inference.
So you can see denseness do perform.
They're faster than resonates and as well as more accurate.
So that's quite an achievement.
So that's it for a tour of all these CNN architectures.
These this should give you a very good grasp and understanding of complicated state of the art scene
and architecture designs and why they work.
So what we'll do now, we'll stop there, even though there's more to this, which was expanded for
the lessons.
I'll talk about things like So we'll stop here for now.
That basically ends the tour of the state of the art CNN architectures.
Not all of these were state of the art.
However, it is very important to understand the foundation of this the designs of these CNN's because
many of them will be utilized in future networks as well as we take bits of Terry here and there that
we've learned.
Researchers have learned that this works.
So it's very, very good that you understand this will also take a tour of vision transformers.
However, I'll do that later on in the course.
For now, though, what we'll do, we'll dive into the code and actually load some of these pre-trained
networks and explore them.
However, these networks are all trained well.
The ones we can load with cameras of PyTorch are all trained on the image networks.
Which brings me to the next chapter of what is image that we may have mentioned.
It's the benchmark image dataset, computer vision dataset that we used to benchmark these models.
However, we haven't really explored it or understood it.
So that's what we'll do in the next section.
And then after that, we'll go back into the code and start working with pre-trained models.
Many of these pre-trained models like Feig and ception mobile net efficient net.
So we'll take a tour of this and give you some hands on experience with loading and unloading some of
the best networks and implementing them yourself and testing testing them on some images.
So I'll see you in the next section.
Thank you.