And welcome to our lecture on our CNN's first hour of CNN's and faster CNN's.
So let's begin.
So firstly, what are CNN's well, not to be confused with recurrent neural networks, so Arnolds are
CNN's were one of the first deep learning based object detectors.
And it gave us very nice results like this output here.
No, they were introduced in 2014 by researchers at the University College of Berkeley and our CNN's
Opt-In dramatically high performance in the Pascal Video Challenge.
That's a dataset that's similar to cocoa that was a standard used back then to evaluate optical detectors.
So but what exactly are our CNN's?
Well, it stands for regions with CNN's or our science for short.
And now let's dig in to the technology behind our CNN's.
So firstly, our CNN's attempted to solve the exhaustive search problem that was previously formed by
sliding windows.
Remember in the sliding window problem.
You can just have an infinite number of classifications, almost because depending on the image size
and what size of box with stride to use, it basically was an exhaustive and slow searching problem.
So how does the algorithm?
How does our CNN's solve this?
Well, they use something called the selective search algorithm, and we can see a quick overview here
of our CNN's.
Basically, we have an input image.
We extract some region proposals that's done by the selective search algorithm.
Then we pass those proposals to CNN to get the features and then we classify based on those extracted
features whether what class it is basically.
So we classify each of the regions proposed here by the selective search algorithm.
So it's simple, isn't it?
Well, let's talk a bit more about the selective search algorithm.
A selective search attempts to segment the image.
So an unsupervised learning method here to into different groups by combining similar areas such as
colors, textures and proposed these regions as interesting bounding boxes.
So you can take a look at this analysis here.
You can see this is a region.
So this is a segmented region here, different types of segmentation as we use different skills and
you can see the bounding boxes that are proposed out of each segmented region.
So it's almost like industrial control over each of these here.
And those were basically our bounding box proposals.
And you can see another example of it here.
So let's talk a bit more about selective search.
So when selective search isn't a fight to regions of boxes, it passes those extracted images to our
CNN.
That's usually a one free tree, and CNN creates an image net of music.
The image?
Not almost always.
And we don't use a CNN directly for classification, although we can.
What we do is we extract the features.
Remember how we used pre-trained models as feature extractors and we just get a vector out of it.
Flattening lastly, the output of the CNN here and we just passed it into something like SVM or logistic
regression, a linear classifier to classify basically.
And now, after our region has been classified, we then use basically in simple linear regression to
generate a title bounding box around the initial proposal bounding box.
So that's it for the initial our CNN's.
However, there were a lot of problems with it.
So while they were quite good, they were notoriously slow as each bounding box had to be classified
by CNN.
And basically this if you get extracted boxes and you have like a hundred boxes, that's a hundred inferences
you're going to have to make on CNN.
So you can see doing this on a video is going to be quite slow.
So the reason for the heavy computation is mostly due to using trees separately.
Train models, we had one for feature extraction.
We had a SVM to predict the final pass and then we had a linear regression to tighten the voting box.
So in 2015, fast as our CNN's were introduced to solve a lot of these problems.
So combining the training of the CNN, the classifier and the bounding box progress into the simultaneous
into a single model was how the researchers were able to solve a number of these problems.
So fast to our CNN's firstly reduced the number of proposals on the boxes by removing the overlap generated.
But how did it do this?
Well, we first rerun the CNN across the image just once using a technique called region and region,
region of interest pooling or our life.
And then our light pool allows us to shift the focus of the CNN for the image across its subregions.
And this works because previously regions who are simply extracted from the CNN feature map and then
pooled.
Now this means that therefore it is only need to run out CNN once on the image, and we can reuse those
extracted features, the extracted from the regions in the areas of the up to the vector.
So we save a lot of processing time there.
So.
Also, training time was improved by combining treating up to CNN Classify and the bounding box aggressive
into a single model.
So our SVM feature classify that we use previously well, that became a soap box layered on top of the
CNN and also the linear regression bounding box type.
No, that just became a bounding box upwardly.
Also a parallel to the soft max coming out of the CNN.
And you can see how drastically fast are CNN's improved training time, as well as the inference time
here.
It was substantial, so this actually allowed it to be used real time in videos.
However, the research is being done.
In 2016, there were more improvements because first our CNN's wildly improved speed.
Significantly, this still relied on the relatively slow selective switch algorithm.
But fortunately, Microsoft researchers found a way to eliminate this bottleneck.
And you can see how much faster, faster RC it ends off a better, faster net recipients.
It's basically a 10x increase in speed, and this one has to do with substantial jumps between our CNN's
Faster CNN's and faster are CNN's.
So let's see how this sped up the original proposal.
So selective search relies on features extracted from an image.
Now what if we just reuse this, which is to do region proposal instead of basically something like
this here where we have the conveyors are putting a region proposal would feature maps here.
We just get proposals out of it and pass those proposals using our way pooling to a classifier.
That was the insight that made faster our CNN's extremely efficient.
So fast to our CNN's added fully convolutional layer on top of the features of the CNN to create the
region proposal network.
So what is of the people actually state?
The region proposal network slides a window over the features of the CNN, so each window location,
the network outputs a score and a bounding box put anchor.
Hence, we have four key where key is no anchors box coordinates because a box has four dimensions.
OK, that's a we have for box coordinates.
Now, after each pass of the sliding window, it outputs key potential bounding boxes and a score or
confidence of how good this box is expected to be.
That's basically it for fast to our CNN's.
So I hope you enjoy this lesson.
And what we'll do next, we'll take a look at an introduction to single shot detectors, also known
as Essos, which are a very cool one shot, which I'll explain in that section type of up detector model.
So stay tuned for that.
Thank you.