Hi guys.
This lecture presents an overview of YOLO algorithm.
Let's look at the topics which we will cover in this lecture.
So following are the objectives.
Or you can say that we have submitted this complete lecture into these six parts which are given over
here.
In the first part, we will see what is YOLO.
In the second part we will see what is basically image classification is in the third part we will discuss
about object localization.
In the fourth part I will discuss about training of a neural network.
In the fifth part I will discuss why we need YOLO algorithm, why we can't use fast R-cnn faster R-cnn
mask R-cnn.
And in the sixth part we will discuss in detail about the YOLO algorithm and its working.
So YOLO is the state of the art object detection algorithm, and it is so fast that it has become a
standard way of detecting objects in the field of computer vision, and YOLO is becoming popular day
by day.
It's gaining popularity.
YOLO.
YOLO was invented in 2015 and since then it's 2023.
Now at eight different versions of YOLO has been emerged.
So the computer vision community has been continuously working on different algorithms and producing
a better version after every few months.
So it can be possible that after few months we can see YOLO being nine and after few months more we
can see Yolo v ten as well, because Computer Vision Community has been continuously working on it and
we are getting better and better results.
Like with Yolo v eight we are getting a mean average precision of 53.7, which is highest ever in the
YOLO history.
So.
So YOLO is has been continuously gaining popularity and a lot of work has been done around that.
On YOLO.
So before YOLO people were using sliding window or sliding window object detection.
Then a more faster versions were invented, which include region based convolutional neural network,
fast region based convolutional neural network, faster region based convolutional neural network and
the Mask R-cnn.
But every each of these algorithm has its some drawbacks and some key takeaways.
We will have a detailed look on R-cnn Fast R-cnn and faster R-cnn in the next lecture.
So from there you will get what are the drawbacks of each of these algorithm.
So YOLO was invented in 2015.
YOLO outperforms all the other previous object detection algorithms.
In image classification, we simply try to classify whether we have the desired object in the image
or not.
Like in the case over here, we are just trying to classify whether there is a deed or a person in the
image.
So you can see that is this a deer or a person in the image?
So if we pass this to neural network, neural network, or you can say convolution neural network,
convolutional neural network will classify as deer is equal to one because we have the deer in the image
and the person will be zero.
Or in other case, if we have a person over there.
Our neural network output will be person is equal to one and T is equal to zero.
So in image classification, our goal is to simply classify whether we have the desired object in the
image or not.
In case of object localization, we are not telling what class basically it is, is it a deed or a person?
But we are.
So we are also telling about bounding box or the position of object in the image.
Where is exactly is that deer inside in the image.
So in image classification will simply tell us that there is a deer or some other object in the image.
So if there is a deer, it will be one.
Or if there is no deer, it will be output will be zero.
But in case of object localization, we are not only telling what class it is, it is a deer or some
other object, but we are also telling about bounding box or the position of object in the image.
Where exactly is the deer in the image?
You can say what exactly is the position of deer in the image.
So this is what object localization is.
In case of object localization, our output will be in the form of like this.
For example, if we have a D in the image, our neural network will output D is equal to one.
And if there is no person inside the image, it will person is equal to zero because we are doing classification
with about D and person.
If there is a deer inside the image or is there a person in the image?
So if we have a deer in the image, our neural network output will be d is equal to one and person will
be zero.
Plus we have the bounding box around the image, around the detected image.
Like you can see, we have detected deer in the image.
So we have created a bounding box around the deer.
So in case of image classification, we don't have the bounding box.
But in case of object localization, we have a bounding box around the detected object, which is D
over here.
Neural network.
We have our output of this form for each detected object.
We have a vector of size seven, which you can see over here.
So here we see represent the probability of plus.
So if there is some object in the image which we want to detect, for example, if we want to detect
a D or a person and the image contain a D or a person, it means there is an object which we want to
detect.
So if there exists an object which we want to detect, then the probability of class will be one.
The probability of the class will only be zero if there is no such object in the image which we want
to detect.
For example, the image is blank.
There is no object in the image, so the probability of class is zero or there is some other object
in the image which we don't want to detect.
The probability of class is will be zero if there is any such object inside the image which we want
to detect, the probability of class will be one.
The second and the other term is B x and b y.
B x and b y.
Represent coordinate represents the coordinate for center, which is indicated in the yellow color.
So this is the center point of my detector object.
So basically B, x and b y over here represent the coordinates of the center which is indicated over
here in the yellow color, like you can see over here.
C1 represents class one, which is for, for example, if C if we are doing classification of deer and
dog.
So what denote a person.
So C1 represents the class one which is for deer.
C2 represents the class two, which is for the person and BW and B represent the width and height of
this red box.
Like you can see over here.
Here, this red box, height and width is being represented by BW and B, so this is basically BW.
And this is basically B edge over here.
So this is our B edge.
And BC is equal to one.
If there is an object in any object in the image, if there is no object in the image which is equal
to zero.
And in that case, in that case, the rest of the values doesn't matter.
So if we have a probability of class zero, then other values don't matter because we don't need these
values.
Okay.
So I hope you got this point.
At the training of a neural network to train a neural network to classify the object as well as the
bounding box.
So as the neural network only understand numbers.
So for each of the image, we will have a corresponding vector of size seven like this, like this is
the corresponding vector of size seven for the digit.
This is the corresponding vector of size seven for the person in the image.
So in the same way we can have ten thousands of image.
So we need to train a neural network in such a way if input a new image.
Now it will tell us that this particular vector in the output.
So in this way, we want to train a neural network.
Ian used to train a neural network in such a way.
If I input a random image at the input at the output, it will give me a corresponding vector like this.
And plus we will also have an output image with a bounding box around the detected object like this.
You can see.
So this is our aim of a neural network.
Like we need to train a neural network in such a way.
If I pass some random image at the input in the output, it will it will give me a vector of size seven
like this.
And through using this vector of size seven, I will draw a bounding box around the image which you
can see over here as well.
So the question you might all be thinking that if neural network works so well for object detection,
then why do we need YOLO?
Because neural networks works well only for a single object like you can see here.
For example, if we have a date or a person in the image, like if we have only a single object, like
there can be a date or a person in the image.
Okay, so if we have a single object in the image, the neural network will perform, give us good results.
But if we have multiple objects, like if we had if we have a date as well as the person in the image,
then neural network will not work.
As as good as it did was for a single object.
So.
In case of multiple objects in an image.
For example, five persons, one dog.
For example, if we have n number of objects, like ten objects in the image, then determining the
size of neural network is very hard.
As I told you, for each object in the image, we have a vector of size seven.
Okay, so if we have multiple objects in the image, the vector size increases.
Like if we have ten objects in an image, then the size of vector will be 70.
Like we have 70 different vectors of an image.
So this becomes computationally very expensive.
So we can draw analysis if we have a single object in the image, like a person, dog, cat, then neural
network works well for object detection.
But if we have multiple objects in the image, so if we have ten objects in the image, then we have
70 vectors.
So.
This becomes computationally very expensive.
So we can't use neural network for object detection.
Plus, if we have 1200s of objects in the image, then determining of the size of the neural network.
Becomes very hard.
So we need to have something that we can have.
We need to have another approach.
So for this we have YOLO.
YOLO algorithm divides the image into grid cells.
So, for example, here I am using a 4x4 grid here.
Like you can see over here, we have four grids one, two, three, four in horizontal direction.
And the four number of grids one, two, three, four, four grids vertically as well.
So we can have this is not necessary that you should have have 4x4 grid.
We can have 19 cross 19 grid three cross three grid as well.
So if we focus on this specific grid like this, this specific grid, so as we have no bonding object
in this grid cell, so the probability of the class will be zero and the rest of the values doesn't
matter.
So if as we have no object in this grid cell, so the probability of the class will be zero and the
rest of the values does not matter to us.
So now we can see that we have a cow and a girl in the image.
We can see that the cow is expanding to two grid cells in width and in terms of height.
The cow is expanding to three grid cell.
So as we can see that.
The call is expanding to multiple grid cells.
We try to find the central place of the cow, and the cow belonged to that grid cell as shown below.
So this is the central place of the cow.
And the cow basically belongs to this grid cell.
So you can have a have done the snapshot of this specific grid cell where the central place of the cow
is.
So here we have a vector for the cow represent the probability of class, which will be one as we have
a cow in the image.
This represents the.
Basically there are these represent the center points.
And here we have to represents the weight as it is covering two cells, like in the width section.
It is covering the core is covering two cells, one, two.
While in terms of height, the core is covering three cells.
One, two, three.
Okay.
So we have two, three over here.
And in the C1, C1 represents the class one, which is for the core.
So we have the core here.
So the class will be one.
And this image does not contain the center point of the girl as it only contains the central point of
the core central place of the core.
So to represent the C2 represents the class two, which is for Blur and it will be zero.
Well, as we have divided our image into 4x4 grid cell.
So total 16 grid cells we have in a single image like you can see over here, we have divided our image
into four cross four grid cells.
One, two, three, four, five, six, seven, eight, nine, ten, 11, 12, 13, 14, 15, 16.
So we have total 16 grid cells in an image.
So for each grid cell we have a vector.
So we have 16 such vectors and each vector length is seven.
So we have 16 such vectors of size seven So for each object, for each grid in the image, like for
each of the grid cell in the image, we have a vector like this.
So for each image we'll have 16 such vectors, like we have 16 such vectors in each image.
Similarly, for this image we will have 16 sets vectors of like this form of this form in the image.
So for each image we have 16 such vectors and for the training we can have thousands of images for the
training as well.
So in this way we will train our model.
Now in the next slide we will look at the prediction part.
After training our model.
If I want to test my model on some sample image.
So this is the sample image which I have passed to test my model.
This is this image.
So in the output I will have 16 such vectors because I have trained my model for 4x4 grid cells where
each grid cell contains a one vector.
So we will have 16 such vectors and the output as well.
Like you can see here, because our output image will be divided into four cross four grid cells and
we will have 16 such vectors of this form of the size seven.
So each vector size will be seven in the output.
But wait, there can be an issue.
The algorithm might detect multiple bounding boxes for each of the object, like you can see here for
the person, we have two bounding boxes, one with the confidence of 0.45 and other with a confidence
of 0.85 for the baby as well.
We have two bounding boxes, one with a confidence of 0.75 and other with a confidence of 0.65.
And for this we have two bounding boxes as well, one with a confidence of 0.95 and other with a confidence
of 0.35.
So if we have multiple bounding boxes for each object, we can take max for each class, like for this
class, we can take the max of these two bounding boxes like we can.
Can't take the 0.95 and remove the other, which is less with less confidence score.
And we can't take Max for this class of baby.
Like we can't take the 0.75 bounding box and remove the 0.65 bounding box.
And for the person we can't take the 0.85 bounding box and remove the 0.45 bounding box.
So when you face this issue, when you have multiple bounding boxes for each of the detected object,
you can't take max, you can't take the bounding box, which has the maximum confidence.
There should be another way, which is IOU.
IOU stands for Intersection Over Union.
Let's go into detail in the next slide.
Solve the issue of multiple bounding boxes.
We use an approach called IOU, which means intersection over union.
So if we only take this class person over here, if we only take this class bounding boxes so you can
see we have two bounding boxes for this class person for only this person.
So for class person, we find the overlapping area and to find the overlapping area we use intersection
over union.
If objects are overlapping, the value of intersection over union is more if they are completely overlapping,
the value of intersection over union will be one.
So for example, we can discard all the rectangle which have IOU greater than 0.65 and keep the rectangle
which have the class probability as maximum.
The same process will be repeated for other classes as well, which means for baby class and for this
object class as well.
And this is the formula for the IOU, which is intersect area over union area.
And in this way I have explained this process like this is our intersection area.
You can say the inner area of the blue lines and this is our union area.
This is this is and this is my union area and this or whole area.
This only area is my intersection area.
Watch because they use IOU to solve the issue of multiple bonding of boxes.
This approach is also named as non-max suppression.
So you can also say this approach named this approach as non-max suppression.
Also, after deducting all objects, we apply.
Non-max suppression and we get the unique bounding boxes in the output.
As we have divided our image into 4x4 grid cell.
So we might sometime face, sometime face this issue, that one grid cell like this has center of two
objects like this grid cell contain the center of the dog and the girl.
So now, as this grid cell above can only represent one class, how we can represent two classes.
So our one grid cell has a vector.
So each grid cell has a vector of size seven, and each grid cell can only represent one single class.
We cannot represent two classes in a single vector of size seven.
So if we face this issue, how we can solve this issue.
Let's look in the next slide.
F1 grid cell has sender of two objects.
So in such case, instead of having a single vector of size seven, we can have a vector of size 14.
So this first seven vectors in here, the first seven numbers over here represent for the first class
and the remaining seven numbers over here represent for the other class.
So if one grid cell has center of two objects, in such case, instead of having a single vector of
size seven, we can have a vector of size 40.
To solve this issue.
Well, thank you for watching this video.
See you in the next video video tutorial.
Till then, goodbye.