In this video tutorial, we will look at how we can train the YOLO 11 object detection model on a custom
data set.
We will train the YOLO 11 object detection model on a Personal Protective Equipment data set, which
is available on Roboflow.
So here is a quick demo or demo of how our results will look like at the end.
After we train the YOLO 11 object detection model on Personal Protective Equipment data set.
So over here you can see that we are able to detect the protective helmet protective boots, safety
vest.
Along with this we will also be able to detect the gloves and the eyewear as well.
So this is a quick demo at how our results will look like at the end.
So let's move towards the coding part.
So over here you can see that we have created a Google Colab notebook.
So I have written all the code already.
So you can see over here we have written all the code.
I have done the training as well.
And you can see we have some results over here as well.
And then we will test the fine tuned model on random images as well as on videos as well.
Okay, so we will cover all this in this video tutorial.
So let's get started with this.
So before we run the script please make sure that you have selected the runtime as T for GPU.
Yes.
Okay.
And now you can see we have the T for GPU available over here.
So Google Colab offers free GPU like you can see over here.
Like it can last around one hour 20 minutes, which is enough for this task.
And we have a GPU Ram of 15 GB available over here and system Ram of 12.7 GB, which is enough for this
tutorial and project.
Okay.
So let's first run this.
So this will tell me that if I am using the GPU memory or not.
So currently I'm using the GPU memory.
Okay.
But no process is running because I'm not training the YOLO model 20.
Then we need to install the Ultralytics package.
Okay.
So over here you can see we are installing the Ultralytics package.
So this will take some time.
Okay.
So now it's done.
So you can just hide the output so your notebook looks more clean.
So now we will check that if we have ultralytics working fine.
So now you can see over here Ultralytics 8.3.1 version is currently running.
Over here we have Python 3.1 available.
And we have Cuda available Tesla t4 GPU.
And we have two CPU cores available of 12.7 GB Ram.
Okay so that looks fine.
So we are using Ultralytics but with version 8.3.1.
Okay.
So now from Ultralytics I will import YOLO.
So when you import YOLO you can just call any YOLO model.
You can call YOLO model over here as well.
You can call YOLO model over there as well.
So now to display any input or output image in the Google Colab notebook we will import the image library
from IPython dot display.
Then I will be downloading the data set from Roboflow.
So let me just show you this data set over here as well.
So we will be using this data set which is available on Roboflow over here okay.
So in order to use this data set uh to fine tune your YOLO model, you just need to log in over here
on Roboflow.
So you can just click over here and you can sign in with Google, with your GitHub GitHub account or
with any other email like Yahoo email as well.
So I will just sign in with my, uh, Google account over here as well.
Okay.
So now you can see over here we will use the version two over here.
And you can see here is our data set.
We have uh, around 3290 images available over here.
And we have around 22,311 images in our train set, uh, 7652 images in our validation set and 327 images
in our test set.
So we have our data set split as like we are using 70% of the data for the training person purpose,
20% of the data for the validation purpose, and 10% of the data is used for the test purpose as well.
Okay.
So this is how our data set looks like.
Okay.
So now I will just download the data set.
Now in order to download this data set you can just click over here.
Then you can select the Yolo V8 format because okay you can either select the yellow V8 or YOLO 11 format.
Okay.
And then you can click on show download code over here.
And then you can just copy this code from here and just add this code over here.
Okay.
So now this will download uh the data set.
If you run the cell over here it will download this data set into Roboflow.
And now you can see that our data set is being started to download.
So this will take some time.
Now it's around.
Done.
Okay.
That looks good.
So now we have downloaded the data set over here.
Okay.
So we have that train test validation data set.
So here is our data dot yml file.
So over here you can see that we have number of classes at seven.
So we have to go to seven classes.
We have the dust mask class, IVR class, glove, protective boots, protective helmets, safety vest
and shield class.
Okay, so these are the different classes that we have in our data set over here okay.
So one thing you can do first before you go ahead is uh here is our train test images and validation.
So you can just copy path from here okay.
And just and just update this path first.
Okay.
And then you can just click over here, copy the path and just update this path over here.
And then you can just click over here Copy Path and just update this path over here okay.
So here is the link of the data set as well.
You can just view this data set by uh copying this link and and adding this into another tab over here
as well.
Okay.
So that looks good.
Okay.
So now if I just right over here, basically our data set is being saved into the variable data set.
And if I just right over here data set dot location this will tell me the location of the data set over
here.
Okay.
So now we will train the YOLO 11 model on a custom data set.
So you just need to, uh, add the model over here.
Okay.
So if you are currently we are using YOLO 11 model.
So you just need to write YOLO 11.
We are using YOLO 11 medium model.
So you just need to write YOLO 11 medium model.
And we will be training or fine tuning the YOLO 11 model on.
Like we can say that 50 epochs because this will be enough okay.
And toss is equal to detect mod is equal to train.
And I'm just passing the location of the data dot yml file over here.
Okay.
And the default image size.
Basically our YOLO 11 model is being YOLO 11 model is being trained on the, uh, image size of six
4640.
So it's better if we fine tune the YOLO 11 model on the same size.
So plus we can like, fine tune it for 40 epochs.
Like that will be fine as well.
So let's run this.
So I'm just showing you like how you can train the YOLO 11 model.
Now you can see it's downloading the data set model over here as well.
Okay.
So now you can see that the training has started.
Uh, so let's wait for the training, uh, to complete, or I will just show you after it completes
ten epochs or so.
Okay.
So let's wait for it to get complete.
So you can see that training is still in progress.
Uh, currently, uh, 30 epochs are being processed out of 40.
And if we just see over here, I made one change.
Previously I was using YOLO 11 medium model, but now I'm using YOLO 11 nano model, which is the smallest
in size and it is the fastest, but it is least accurate among the YOLO 11 models.
The reason is that because, uh, to process an epoch, the YOLO 11 medium model was taking around,
uh, 3.5 minutes.
Uh, so it will take a lot of consumer, a lot of time.
So I just shifted to YOLO 11 nano model, which is taking around 53 seconds to process an epoch.
Okay.
And currently, uh, 30 epochs, 31 epochs are being processed.
So let me try to explain you a few things in the meanwhile.
So now over here you can see that we are using YOLO 11.
We can just close the dataset that we are using.
Uh, these are the different parameters.
We are using the YOLO 11 nano model.
And we are using the we are just training or fine tuning our YOLO 11 nano model on this Personal Protective
equipment data set.
We have set the epochs to 40.
The patience is being set to 100.
Like if we are getting, uh, the mean average precision like this is the mean average precision with,
uh, IOU threshold, uh, intersection over union threshold as 50 or 0.5 is 0.421 and mean average precision
when IOU threshold.
Intersection over union threshold varies from 0.5 to 0.95.
Is 0.238.
So the patient's value is being set to 100.
So if the mean average precision.
Um remains same for 100 epochs, then uh we will stop the training.
Okay.
So this is what.
Patient will reflect then we have a batch size is equal to 16.
Like in each batch we will pass 16.
Images to our model okay.
So for training uh, in each batch we will pass 16 images to our model.
For training.
So what does this mean.
So these are the number of batches 142 are the number of batches.
Okay.
So in each epoch we pass 142 batches okay.
So like you can see over here and in each batch.
We have 16 images.
So if I just show you over here uh number of images.
So we have 2271 images in the training.
Okay.
Like you can see over here, we have around 22,271 images in the training.
You can see over here.
Okay.
So what we can do for here is if you divide this number with 16.
Okay.
So now let me see if I just open the calculator from here.
So as I told you that in each batch we are passing 16 images.
So if you divide 2271 divided by 16.
So like you can see over here, these are the number of batches.
So what we have 142.
And if I just show you over here.
Like you can see over here we have 142 batches as well.
Okay.
And over here in the validation we have around.
in the validation.
We have around 637 images like you can see over here.
Okay.
So if I just divide 637 divided by 16 is equal to around 40.
But you might notice over here one thing that we are have we are the number of batches should be 40.
But the number of batches over here are 20 because we have reduced the size of the batches.
So you can say that we are just passing 32 images in each of the batch over here in the validation.
Okay.
So and if you just go over here we are setting, we are just fine tuning our YOLO 11 model by setting
using the default image size.
Basically YOLO 11.
Our model is being trained on that by using the image setting.
The image size of 640 cross 640 like 640 is the width and 640 is the height.
And we are just fine tuning the YOLO 11 model by setting the same image size of six 4640.
So if our image are larger than six 4640, then it will be compressed to six 4640.
So it is better if you don't, uh, change the values of the image size and keep it default.
Because if you just vary those values, like if you increase the image size or decrease the image size,
the performance of the model get might get compromised as well.
Okay.
So we have the batch size 16.
And uh let's and in the train three folder our results will be saved okay.
So if I just go to the run dataset.
So in the train three folder we have our weights and everything okay.
So we are using the pre-trained model.
So it's true we are using the optimizer automatically defined okay.
Uh verbose is equal to true because you can see we are just getting the detail.
Like after each epoch the mean average precision we are getting.
So we are just getting all these details.
So this is because we have set the.
By default the verbose is being set to true.
Okay.
And if we just go ahead.
Um.
Okay.
So.
And then we have uh, so these are all the other parameters.
You can just change these values.
Parameters over here.
These are the by default set values.
Uh, I was just trying to show you one thing.
The maximum number of detections.
Okay.
So you can see that, uh, the maximum number of detections are being set to 300.
So it will detect maximum 300 objects in a frame or in an image.
Okay.
And this is the non-max suppression intersection over union value is being set to 0.7 okay.
So.
So basically non-expression is a technique that is used to remove redundant or overlapping bounding
boxes.
Um, so all the objects that have an intersection over union, like uh, below 0.7 are being removed
or treated as false positive.
Okay.
So if the intersection over union of the two bounding boxes is below 0.7, uh, then the bounding boxes
which have a low confidence threshold will be suppressed.
So this is what we mean by IOU over here.
So and uh, then all these are all the other parameters.
So this is what we need to know.
So let's see where our training has progressed for now.
So we are done with the 37 epochs.
Okay.
So this will take around two minutes more.
So let's wait for this to finish.
And you can see over here we are just getting mean average precision with intersection over union threshold
as 0.5 or 50% is 89.8%, and the mean average precision with intersection over union threshold varies
from 50 to 95, or 0.5 to 0.95, is 0.617 or 61.7%.
So these are all the calculations that are obtained on the validation set.
So basically we are not using the validation images like we have uh 637 images in the validation.
So we are not using the validation images for the training.
So these are all the mean average precision scores on the validation data set okay.
So.
So as we are not uh training our model on the validation data set so we can just use the scores and
see how our model perform.
Okay.
So basically uh, the mean average precision score on the validation data set tell us how better our
model generalizes.
Okay.
So now you can see that training is being done.
And you can see that, uh, we are just getting the mean average precision scores, uh, for all the
classes, uh, these are the scores that are obtained on the validation data set.
And our best model are saved into this, uh, train tree and, uh, test.
So these are our best model weights over here.
We have.
Okay, so now you can see that the mean average precision score for only shield class is low, like
it's 0.763 and mean average precision when IOU varies from 50 to 95 is 0.467 else or for all the classes
it's much higher.
Okay.
And the precision and recall scores are also a bit low.
Okay.
The reason might be that, uh, for the shield class, we don't have very much data available like an
export in the training as well as in the validation.
Like you can see over here, we have uh, in the validation, we have 637 images and we have only 25
images for the shield class.
Similarly, in the training, we have also very low data for the shield class as well.
Like if we increase the data might be, uh, we might get better results.
Okay.
So now if I just go to over here we have the precision recall curve the confusion matrix.
So let me just show you the precision curve first over here.
So one thing you might notice that um as the confidence value increases uh, now you can see what is
precision first.
Uh, precision is basically uh, number of true positives.
Like how much correct position predictions we got out of all positive predictions, like true positives
divided by true positive plus false positives.
So in short, you can say that precision is the number of correct predictions like we got.
Okay, so now you can see that as we increase the confidence threshold the precision value continues
to increase okay.
Now you can see that as we increase the now you can see we have the for each class we are just getting
a separate precision curve okay.
And as we increase the confidence threshold the precision value is increasing.
Like you can clearly see over here for all the generally for all the classes.
So this means that at a higher confidence threshold the model is making very low false positive predictions.
As we increase, the confidence for the model became more certain.
And although it might make very less predictions but it will make a correct predictions.
Okay, so as we increase the confidence threshold, the model makes a very less false positive predictions
we can clearly see over here.
While we can see that when we uh, when we have a low confidence score like 0.0, um, we have uh,
very low precision score.
This is because we are just getting many false positives in our predictions.
When we get more false positives in our prediction, our precision value drops.
And when we get, uh, no true, no false positives, like when we increase the confidence score, we
definitely don't get, uh, very much false positive.
So our, uh, precision increases.
So this is how it relates.
And you can see over here this blue curve is for all classes.
And we can see over here uh at 0.959 confidence threshold, we are getting a precision score of 1.00.
So this means that 0.959 confidence threshold.
We are getting a precision score of 1.00.
And here we have the recall confidence curve over here.
So we can see that the recall value for all the classes drops as we increase the confidence scores.
Like you can see at low confidence, uh, we have uh at low confidence score, the recall value is high.
And as we increase the confidence score the recall value drops.
So, uh, basically, uh, recall tells us the true positives out of all ground truths.
Okay.
So what does this mean?
Like when we have a low confidence score, like, uh, in recall, we are not worried about false positives,
okay.
Only in recall we are worried about the true positives.
So when we have a low confidence score like 0.0 we are getting more predictions.
And in general, when we have more predictions that we might have very much high false positives.
But we also have many true positives in our predictions as well.
So in recall, we are only concerned with the number of true positives okay.
So as we decrease the confidence score we might get a lot of true positives.
So we have a high recall at low confidence.
And uh when we increase the confidence score like 1.00 our we might not get uh um, we might not get
very much predictions like the prediction rate drop.
Like if we have the confidence score 0.0, we are getting, for example, ten predictions.
And when we make the confidence score 1.0, we might be getting, uh, around 1 or 2 predictions.
So when the prediction rate drops.
So it might mean you can say that we might get less true positives okay.
So therefore the default score also dropped because we are getting less true positives.
So because when we increase the confidence score like 1.0 or 0.8, there will be less detections, okay,
because it will only keep the draw bounding boxes around those detected objects which have a higher
confidence above 0.8.
So definitely the predictions will drop.
And we might have uh, although we might get low false positives, but we all we are also getting uh,
very much low true positives as well.
So therefore our recall score will drop.
But one benefit is that when we increase the confidence score, you might get, uh, fewer predictions,
but they will be more accurate predictions.
So remember this.
When you increase the confidence score you might get fewer predictions, but they will be more accurate
predictions.
Okay.
So this is the confidence score.
So the confidence score basically tells us how our models handles different classes.
So for example for the dust mask class, like you can see that um, at 282 times our model detected
correctly that there is a dust mask, while 16 times when there is a dust mask our model was unable
to detect, like our model detected nothing.
And 128 times when there was, uh, where the user was wearing an eyewear.
Our model detected correctly that the person is wearing an eyewear, while one time when the person
was wearing an eyewear, our model detected it as a shield.
And 18 times when the person was wearing an eyewear, our model detected nothing.
Okay, similarly, 725 times when the person was wearing protective helmet or model detected correctly,
that the person is wearing protective helmet and uh, 23 times when the person is wearing protective
helmet, our model is unable to detect anything.
Okay.
And one time when the person is wearing protective helmet or safe shield, and two times when the person
is wearing protective helmet.
The our model detected as safety vest.
So confidence metrics tell us how our model handles different classes.
Okay.
So you can just run this.
There will not be very much difference.
Okay.
That's fine.
Okay.
So now over here now using these are the normalized confusion matrix.
Okay.
So now you can see 93% of the times our model correctly detected correctly that there is a dust mask.
While 7% of the time our model are detected incorrectly or our model was unable to detect anything.
Uh, 80% of the time, our model detected correctly that there is an IVR, and 1% of the time when there
is an IVR, our model detected it as a shield.
So this is how we can use confusion matrix.
Okay.
And then this tells us how much data we have in our data set.
Like you can see over here for the shield class we have very much low data like very less data for the
IVR.
We also have less data.
But for the, uh protective boots, protective helmets, safety vest, we have huge amount of data like
these.
We have 2000 instances around for the glove.
Okay.
So now you can see these are the results over here.
Now we can see that, um, the mean average precision is increasing.
We have loss decreasing.
So this is how these are the model predictions on the training batches okay.
And these are the model predictions for the validation batch.
And this looks good.
Like model is able to detect protective boots gloves.
So we have not used the validation data for the training.
So it's always better to take a look and see how our model is performing.
Okay, so now what I can do over here is these are my model weights.
So one thing I can do over here is I will just save this model weights into the drive.
So now what I will do is I will just right now.
I just need to mount Google Drive in the Google Colab.
Uh, yes.
That.
Yeah.
So I can just do is I can just copy this from here.
And I can just run this over here.
I will just connect to the Google Drive now.
Okay.
So this will take some time.
In the meanwhile, I will just open a drive over here.
This is my drive.
Okay, so if I just refresh this, we have the drive over here.
Okay.
Okay, so one thing I can do is I will just, like, move.
And I want to switch.
I want to move this weights.
Okay.
From there to, uh, in which folder I want to move this, I can just, uh.
Oh.
This here.
Okay.
So let's see.
And now if I just go to Yolov7 over here, I should have best spot.
Yeah.
Where we have our weights.
So one thing I can do over here is.
So you can just copy this thing from here.
Oh, we only need this link.
Like and share.
So what we can do is you can just delete this cell.
So let's see if it works on.
So now you can see we are able to download the model weights.
And you can see we have our best model weights over here.
Okay, so now we'll just pass those best model weights.
And we will now validate our fine tuned model.
So we'll see how our model generalizes.
Okay so.
So now you can see over here we have our 637 images.
And we are just passing those images in a batch of 16.
Okay.
And so we have total number of batches as 40.
And you can see over here only for the shield class we are just getting a bit low mean average precision.
As for all the classes we are getting good precision recall and mean average precision score okay.
And our results are saved in over here.
Okay so we have the confusion matrix F1 curves all those.
So now we will just test our model or do perform inference.
Or with our fine tuned model on the test data dataset images.
Okay.
So does it mean that we have these images and we are just trying to perform inference on this test dataset.
Images.
So we have around 322 images in our test dataset.
Okay.
So it will take few seconds.
So now I will not display all these images over here.
I will just try to save like three images over here.
And let's see how our results look like.
Like basically these are the this is the unseen data.
We have not trained our model on a test data set.
So our model is able to detect glove or dust mask, helmet, uh safety vest glove, protective boots.
Results.
Good.
Dust mark glove.
So now you can see that our results look quite promising over here.
Okay.
And now I will just download a sample image from my drive and see how our model performs.
Okay, so I'm just setting a confidence threshold of 0.25.
And I want to save the results.
And let me just display this results over here.
So now you can see we are able to detect the safety vest protective boots, safety vest protective helmet.
Like we are just getting good results.
So now I will download, uh, two videos from my drive directly into this Google Colab notebook so that
I can test this fine tuned model.
Okay.
Okay, so let's in the meanwhile let's run this up as well.
So now our the complete video is being divided into 310 frames.
And we are doing detection on each of the frame one by one.
And in the output we have a complete Update video.
Okay, so let me just display this output video over here.
So this will take a few seconds before we have the output video being displayed over here.
Uh okay.
But you can see our output video is basically saved over here.
This is our output.
Now you can see the size is 92 MB around.
So it will just compress the size like you can see over here.
It's around 3.7 MB.
And now we will display this output video over here in Google Colab notebook.
So this will take some time more.
And where we have the output.
So here is our output video in the Google Colab notebook.
And let me download this video and show you how the results look like.
Okay.
Now you can see over here we are able to detect the safety valves protective protective helmet.
The results look quite promising.
Like you can see over here, we are able to take safety vest, protective helmet, protective boots.
Okay, these results look quite promising to me.
Like, very good results.
Like, you can clearly see over here.
Uh, in the same way we can test our model on this other video over here as well.
So now you can see the complete video is divided into 267 frames.
And we are doing detection on each of the frame one by one.
I'm just saving down the notebook.
Okay.
So let's run this over here.
Okay.
So now we are just, uh uh, our output video is being saved to over here detect predict for.
And I'm just trying to display the output video over here.
Just Google Colab notebook.
So this will take few seconds.
So in short, in this tutorial we have learned that how we can fine tune or train the YOLO 11 model
on any on personal protective equipment data set.
You can also train or fine tune the model on any other data set as well, and our results look quite
promising and clearly see over.
So this will take few seconds before we have the output over here.
So here is our output video.
Let me download this video and just open this video now.
So over here you can see that we are able to detect the safety vest protective boots uh helmet protective.
So our results look quite good.
In this tutorial we have seen that how we can train or fine tune, uh, the YOLO 11 model on any custom
data set.
That all that is all from this tutorial.
Thank you for watching.