Welcome back to the course in this lesson, we'll take a look at efficient net, and I'll show you how
efficient it improved accuracy and efficiency.
True.
Google's auto ml and model scaling techniques.
So let's get started.
So efficient.
That was introduced in 2019 by researchers at Google and the main motivation behind Efficient that was
that the researchers wanted to figure out a way how do we optimize the architecture of our networks
because CNN's are typically designed at a fixed resource cost and then it scaled up?
This isn't the best way of doing this.
You actually can waste a lot of time and efficiency.
So we need to figure out things like how do we increase steps like number of layers or wit number of
filters?
How do these things affect our accuracy?
How do we design an experiment that optimizes the architecture of our CNN's so that we get the best
results?
So the posited that there must be a principled method of scaling up CNN's.
So that's what efficient net does.
This method uniformly scales each dimension with depth to resolution, width to fix out of scaling coefficients.
So imagine it's looking at like a grid search and a searching for the best parameters in optimizing
the architecture.
It utilizes Google's new AutoML, which uses efficient net behind the scenes.
It's also a way of optimizing to provinces of any existing CNN.
So it's it's a very, very cool way to get better performance from your custom CNN or another CNN you
wish to use on your dataset.
So they were able to surpass the best state of the art networks and achieve a 10x better efficiency,
which is remarkable.
So let's talk about scaling.
So researchers systematically studied the effects of scaling up different dimensions and those dimensions
I'll discuss in the next slide.
And it was found that balancing scaling in all dimensions actually resulted in the best overall performance,
which intuitively makes sense.
But you wouldn't have known this unless you did the experiments and figured that figured out in a principled
way to scale these networks.
So how does efficient net actually determine the best parameter, the best CNN design?
So the methodology is actually quite simple.
What we effectively are doing here is a grid search, and what we do is that we fix the other constraints
here.
So imagine we have these four constraints.
We're looking at number of channels, which is like the depth of the image color of grayscale, the
width.
That's a number of filters.
The depth, which is a number of layers and a resolution scaling, is the input resolution because typically
we use quite small input resolution images.
However, we can experiment by scaling that parameter as well.
So by doing this under a fixed resource constraint and they use flops as the computational resource
resource constraint, this allows us to determine the best or most appropriate parameter for each of
these four.
And then once we do that well, once we do the channels and width, we can actually start scaling up
depth and resolution as well to get the most appropriate or best performing CNN for this specific computational
constraint.
So let's talk about this efficient that architecture now, firstly, as you can see, efficient net
follows a principle.
It's a principle of optimizing the CNN architecture.
However, it depends heavily on the baseline network, which means it doesn't always work, but it worked
fairly well with mobile net and resonate.
And what did Google researchers did was that they allowed efficient, that efficient that principals
to develop basically a new CNN architecture that optimizes both accuracy and efficiency.
And this comes out of Google's utter emails amnesty framework, which was a predecessor for efficient
that amnesty.
Many of you may have heard of that on this if you're in research into research or have looked into Google's
auto email.
So this this new this new architecture design was actually able to develop new networks.
And that's that's what led to mobile night Vision two, which was slightly larger but performs very
well.
So this here is one of the efficient that architectures.
And if there's as many efficient nets it actually you'll see in the next slide here.
This performance is be zero to be seven and is actually a lot of other combinations.
But researchers initially started with these seven networks and you can see they perform quite well.
This is the performance on the y axis here.
You can see the image now, top point accuracy, and you can see if you should not be seven, which
is the deepest backbone achieves any for just over 84 percent accuracy with just over 60 million parameters,
which is quite good.
The networks are a 16 billion parameters previously like maybe Inception Resonant Fusion two and this
Meebo net they understood, which has a bit more is still performed less got less accuracy than efficient
net.
So you can see it every parameter account efficient and is a better network.
So we'll stop there.
And next we'll take a look at denseness.
So I'll see you in the next section.