Hi guys.
In this video tutorial we will see how we can use Adobe Air for personal protective equipment detection.
So the dataset I will be using in this project is available on Roboflow.
Publicly, the dataset consists of 3235 images and let us show you the dataset.
So we have seven different classes which are marked as zero 11 three four, five.
We will change the name of this classes.
For example, in the class number zero, it's handmade.
So we will change the name of the class zero to help it.
The Class 11, which is the shield.
You can see here the person is wearing a shield.
So it's detecting that the person is wearing the shield.
So we will change the name of this class from 11 to shield.
And in the third class, we can see that the person is wearing a jacket.
So we will change the name of this class from three to jacket and plus we can further we can see the
class for the model is just detecting the boss.
So we will change the noun name of this class from four to mask.
In the same way we will change the name of other classes as well.
So like in this case we can see that in the ninth class the model is detecting that it's the class of
boots.
So we will change the name of of this class from nine to boots.
Okay.
So in this way we will rename all the classes so that when the model detects handmade jacket or boot.
So we should know that the model has detected a jacket handmade or both instead of it should appear
in the detected object.
It is zero one 2 or 9.
Okay.
So we don't want a numeric number.
We should we want the name of the class.
Okay.
So let's start towards the implementation.
But before going towards the implementation, let us see the model of health check.
Well, we can see that we have a total of 3235 images and we have each image is contains around in average
4 to 5 bounding boxes.
So in each image we are detecting 4 to 5 different classes or objects.
So we have around 14,341 invasion in 3235 images, and each image contains around 4 to 5 bounding boxes.
And that image size is average.
Size is 4416, cross 416.
But we will resize the image to 640 plus 640 because our YOLO V8 model is trained on 640 cross 640 images,
so it's always better to keep that size.
And here we can see that the class balance so we can see that these three last three classes are unbalanced
so that that data set is not very much balanced.
You can see that some classes are imbalanced or you can say that some classes are underrepresented,
which include class four, six and 11.
Okay.
So if the data set, we can see that we have 2300 images for the training set, 647 images for the validation
set and 324 images in the train testing site.
So our split ratio is 70, 2010.
70% is for the training, 20% is for the validation set and 10% for the testing set.
Okay.
So to import this model.
Into our basic CoLab notebook.
We will click on download and just show down the stacked format.
So basically we will select YOLO V5 PyTorch because Ultralytics has also basically invented Yolov5 PyTorch.
So if we choose Yolov5 PI torch or V8, it's the same.
So.
So just click on show, download ebook and just copy this from here and just click and copy.
So now we will just paste this code into our Google CoLab notebook.
And in this way we will export this dataset from here into our Google CoLab notebook.
So before running the script, please make sure that you have selected the runtime as GPU.
Okay, now we will click on import OS, basically import OS because we are using OS to create a helper
variable over here so that we can navigate to different files dataset easily.
We're just running this cell first, then we are using Glob is used to return all flight paths.
Like if you want the input images file path, we can easily return using glob library.
Then we are importing image and display.
Basically, we are using these two libraries to display any output image like predicted output image
or any input image into our Google CoLab notebook.
So it will display any output or input image into our Google CoLab notebook.
We require image and display library and using display dot that output function we are basically this
function is used to clear output in the notebook.
So if you want to clear output in the notebook we use via dash output function.
Okay, So just okay, so we can leave it for now as well because.
Because currently we don't have any output, so we don't need this.
So in the first step we need to check whether we have access to GPU or not.
So we'll just run this cell.
And it will show us that whether we are using or not so GPU memory uses.
So that's fine.
So we are defining our basically run directory here.
So this is our home directory which is over here.
This all is our home directory.
So now we will install a text using PIP store.
Basically it can be installed in two ways from the source phone or via PIP.
So V8 is the first version of YOLO, the first iteration of YOLO, which has its own optional package,
then YOLO.
V7.
They don't have their own official package, so YOLO behaved as its own official package.
So use installing ultralytics.
We can install YOLO and if you use PIP install ultralytics it will install the V8 version.
So if you do PIP install ultralytics will install the YOLO V8 version.
So there is another way you can install or implement YOLO.
V8 is to clone the github repo of your ultralytics yolo v8 GitHub repo.
So if you want to clone the implement using by cloning the GitHub repo, you can also do this.
Here is the code written to clone the GitHub repo.
So what will follow the easy way, which is the easy way is use to install ultralytics.
We use to clone the github repo in that cases where we need to make some changes in the code.
For example, I want to add the speed estimation in the script in my prediction.py file so that in that
case I will go on the github repo where I need to make some change in the predict.py with a train.py.
In this case I don't need to make change in the predict.py or the train.py or the validation script.
So in the current step straight in the current project we are not making any change in the predictions.
We have training, script or a validation step.
So we are not cloning the GitHub repo.
We need to add some code in the training validation or prediction step then will clone the GitHub repo.
So I'm using PIP install activities to install the latest version of YOLO, which is YOLO V8.
So this will install the latest version of YOLO, which is YOLO V8.
So it might take some seconds.
Okay.
So now we will import ultralytics to check whether our YOLO model installed and it's working fine or
not.
If it's not working, fine.
And see what's the issue.
But it's working fine over here.
So.
Okay, now.
Now we will import the data detection dataset.
This dataset from Roboflow.
So first we will create a folder over here.
So what choice is click on new folder or.
So create a folder here, but we will create a folder using mkdir and the name of the folder is datasets.
Okay, so you can change the name of the folder as well.
So now you can see over here we have a folder by the name of datasets.
Okay, this is an empty folder, but we will download the dataset into this folder.
Okay.
So just seeing what is our present working directory now setting the current directory as this datasets
folder so that we can download the dataset directly into this datasets folder.
So just copy this from here and just remove this and just paste this over here.
So now you will be able to download the personal protective equipment of dataset from Roboflow to your
Google CoLab notebook.
So the dataset is being downloaded from here, so it might take some time.
Okay, so.
It's downloading and it's around 6%, 7%, because that is consist of around 303,255 images.
So it will take some time to download, so please bear with me.
And this is the datasets get download.
So it's at 42% currently and 5,450% 5,560%.
So in this way we will download our dataset completely, but it will take some time.
Okay, it's 90 and it's 99%.
Okay.
The dataset is downloaded over here and we can see, okay, so if you happen some slide, you then just
need to reload this because if I click over here a lot of files appear, so it just need to reload it
and it will work fine.
So don't need to worry in any case.
Okay.
So just wait for few seconds and here is our datasets folder and here is our detection datasets which
consist of training, test and validation.
Okay?
And this is our data file, which consists of nine different seven different classes.
Okay.
One, two, three, four, five, six, seven different classes here.
Okay.
So first we just need to do one thing.
I have told you that we will rename this classes by their names of the object, so I have done this.
So let me upload that updated file data dot yml file over here.
So just give me a minute.
I will upload this.
So, guys, you can see over here, this is my updated data dot yml file.
So you can see that we have a class name against each class, like protective helmet, shield jacket,
dust mask, eyewear glove, protective boots.
So it's not a number.
We have a class name against each class.
Okay, so we have downloaded the data set and just checking we the data set location over here.
Okay.
So.
So basically, if you want to train, validate and run inference or model and you don't need to do any
modification in the code, for example, don't need to add any speed estimation or tracking or any other
step, then you use command line interface is the easiest way to do it.
So we are currently using the command line interface to implement the training, validation and testing
of our model.
Okay.
So if you want to do detection, you can select cross is equal to detect.
And if you want to do classification, you can select cross is equal to classify or segmentation.
You can select class is equal to segment.
But we added to doing detection over here.
So we have selected class is equal to detect and what is equal to train because we have first training
our model and here we have chosen the YOLO V8 medium model and so on.
Okay, so.
Before we start the training.
We just need to make few changes over here.
We just need to rename this folder and just click one enter over here and just open the data dot yaml
file.
Okay.
And just open the data dot yml file.
And just.
Just go to train and just copy part.
And just paste it over here and just go to valid copy this path.
And just paste it over here.
Okay.
And just save it.
Okay, so just click one ctrl s.
And now we are training our model for 90 parks and we are taking our image size at 640.
And here is our data dot yml file path over here and let's run the training.
So first we are downloading our model.
So the training will take around 2 to 3 hours, but we will stop the recording and we will be back when
the training completes.
Okay.
Well.
The training has started.
You can see over here.
The training has started, but it will take quite some time.
So we will pause the video as the training gets started so you can see that the training has started.
So as the training complete, I will be back and explain you the rest of the code.
Till then, see you then when the training completes.
Guys the training of the model has completed.
We have trained our model on 90 epochs with image size 640.
And here are the training results we get.
So in total, we have trained our model on 90 epochs.
So it took around three hours for the training to complete.
And we get we have got the best weights file and the 90th epoch last weights file as well.
Okay, so we have seven different classes, protective helmet, shield jacket, dust mask and gloves
and protective boots.
So here we got the mean average precision with IOU 50 and mean average precision.
When IOU varies from 50 to 95% for each of the class.
So we can see that we have a very good for all the classes like it's around 97.8%, 66.1% and it's 92.8%
for the jacket class and for 96.8% for the dust mask and IOU.
When IOU varies from 50 to 95 with mean average precision.
So we can see that here.
We have also got very good results as well.
So we have a stored our results in program runs detect train to.
And here we can see that here is the word weights file.
And here we have the F1 curve to check what different files or results we have in this folder.
Just run this cell.
So you can see that we have the confusion matrix.
So confusion Matrix basically tells us how our model handles different classes.
We have the F1 curve, precision curve, precision recall curves, and then we also have the recall
curve, and then we have the results in the form of CSP.
So results dot csv file show the performance of the model on each of the epoch in the results dot PNG.
We have the training and validation losses and then we have the model predictions on the validation
batches as well.
Okay.
So let's first see what the confusion matrix we are getting.
So confusion matrix is the chart that shows how our model handles different classes.
So if we consider a jacket, for example, so we can see that 92% of the time our model detected correctly
that a person is wearing a jacket, while 1% of the time we get the bounding box.
But the jacket is incorrectly classified as the ideal, while 7% of the time our model will be trained
fine tune.
Your model for detection is unable to classify that the person is wearing a jacket or model is unable
to detect that the person is wearing a jacket.
So a model does not detect anything, although the person is wearing a jacket.
Okay.
So you can see that while 7% of the time when person is wearing the jacket, the model is unable to
detect it.
So in this way we can get information from the confusion matrix.
Here we can see that 96% of times our model has detected correctly that the person is wearing a helmet,
while 4% of the time when the person is wearing a helmet, our model is unable to detect the helmet.
So here we have the graph with the training and validation loss.
So we have the graphs of training and validation loss in the results dot PNG.
So we can see that the loss value is continuously decreasing.
So if we train this model on 200 to 50 epochs, we can get some more better results.
While you can see that mean average precision is continuously increasing and recall is also getting
better.
So here you can see that mean average precision curve is continuously increasing with 50 and 50 to 95.
So our results are quite good.
So here are the model predictions on the validation batch.
So these images are not used for training.
So it's always better to have a looks like here model is just detecting that person is wearing the dust
mask here.
The model is detecting that the person are wearing gloves on both of his hands and the person is also
wearing protective boots as well.
So the model is working quite fine.
Like here is not wear a glove.
So here's the model has not detected that a person has wear a glove while the person has.
Here we have been wearing a glove.
So model is able to detect that the person is wearing a glove.
Okay, so let's validate the model or validate our custom order.
So we are taking the best weights which we got over here, best dot, and we will validate our custom
model.
So similarly as before, we are using command line interface to do the training.
So we will be using command line interface to validate the model.
So previously we utilized light mode is equal to train.
So here we will write mod is equal to validate and we are performing detection.
So DOS is equal to detect.
And here we are just passing data dot yml file path, which basically data dot file contains our training
set, testing test and validation test images path.
Okay, so here we are validating our custom model so we can see that we also got very good result in
terms of mean average precision with 50 and mean average precision.
When IOU varies from 50 to 95.
So all the results are quite good.
So here we are doing inference machine with custom model.
So inference means a prediction that we can run on image to detect a label, whether it is classification
or a bounding box or a segmentation.
So here we are testing model on the test dataset images.
So here I pass that path for the test dataset images for example.
Let me show you.
Here we have the data set and here we have the test images.
So we have just copied this path and just paste it over here.
Let me do it again.
There have just.
Wait.
I have just added a part of my test dataset images and this is the path of my best weights file which
am added over here.
You can see and I am not doing prediction, so I am just doing more to prediction.
Previously I was doing validation.
So I've written where when I was doing training.
So I was written train and the task is detect.
So now we have been detection of object detection.
So I have written the text detect, okay, so just run this cell and it will throw test our model on
that test dataset images.
So it might take few seconds to run.
Then we will see what the results do we get.
So please wait for two seconds until it drains completely.
Then we will see what results we are getting.
So the model has run on the test dataset images.
So they are 324 images and the results are saved in the prediction.
Predict eight file.
As the results are saving run detect predicted.
So just copy this path and just paste it over here.
So as we have total 324 images, so we will not display all the 324 images.
What results do we get on all the 324 images?
Instead, we will only check the results on the first five images just sitting over here.
So here to display output on the Google CoLab notebook I have imported from IPython dot display import
image display.
So just run this cell now and see what results do we get.
So it might take few seconds, so please wait.
So here you can see that our model is able to detect correctly the protected boots, the jacket, the
gloves, dust masks, the protected helmet.
So results are very wonderful here.
The model is also able to detect the jacket, protective helmet.
And here the model is also able to detect the protective helmets which are all below the gloves, everything.
And here the model is also able to detect the dust mask and the protective helmet as well.
So the results of the model are quite impressive.
Guys, let's test our model on some demo videos and see how our model performs on the demo videos.
So here I am downloading a demo video directly from my Google Drive.
So just run this cell.
The name of that video is demo dot MP4, so it might take few seconds to download.
The video is downloaded.
So now let's test our model on this demo video.
I have passed my best model weights over here.
I have set the confidence to 0.25 cause we are performing object detection and we are doing prediction.
And here is that demo video path.
So just run this cell and see what results do we get.
So it might take few seconds to run this demo video.
So because it will process the video frame by frame so we can see that the model is detecting the protective
helmets, jackets and thus frame rate is 27.2 millisecond.
Okay.
So it's 70, 80, 90.
This is a progressive thing.
We are using GPU so it's a bit fast than CPU CPU in CPU.
It takes quite some time to put to training and to do prediction.
So it's always better to do a test.
Our model on GPU and Google CoLab offers some free GPU as well.
So I'm using Google CoLab free GPU.
Okay guys.
So.
Might take some more time, although we can see that the model is detecting and protective helmets as
well as jackets.
So.
Our results will be saved in runs the.
And train on him.
The prediction too.
And this is our output demo video.
Okay.
So.
As this completed.
Okay, so our results are saved in runs.
Detect predict.
True.
So as our model, this file is a bit large.
So I think it will not be able to display here.
So it's better to download this demo video from here and see what results do actually we get.
Okay.
So it might take some time to download.
So I will pause, pause this video as it downloads.
I will be back and then we will check what the output do we get.
It was able to download the demo video and let me play showing the output.
So our model is able to detect the jacket's protective jacket, the protective helmet.
You can see that the both person protective helmet and the jacket is rejected, although the model is
not detecting the gloves, but the model is able to detect the jacket as well as the protective helmet.
So let's test our model on some other demo videos and see what results do we get over there.
So here we will download the demo video to and see what results we get on that test demo video.
So let me test this.
Our model on this demo video to.
Actually, the name of the video is Demo three.
So let's run this.
So it might take two seconds to execute, but we can see that the model is detecting two protective
helmets, one jacket.
Okay?
Two protective helmets, one jacket, two protective helmets, one jacket.
It might take some few minutes to run.
And then we will download the output and then we will see what results we get.
Our output will be saved in the prediction three And this is the name of our output video demo three
dot mp4.
Okay, so I will just download from here.
So it might take some time to download as the download complete.
I will be back with the output video.
Well, guys, I was able to download the output demo video and let me play the output demo video so
you can see that the model is detecting protective helmet jacket.
And here also the model is detecting the jacket, the protective helmet.
The persons are not wearing the gloves, so the model is not detecting it.
So let's test our model on the demo video three and see what results do we get over there.
Okay.
So let's test our model on the demo three.
So I'm actually not displaying the results over here because these file sizes are large.
So if not, we will not be able to show this output.
Demo videos over here.
I have already tested it, so not wasting time to show it, trying to show the results over here.
So downloading the demo video three from my drive directly into the Google CoLab notebook.
So it might take some time and then I will run the model on the demo video three and see what actual
results do we get?
So it might take some time.
So.
Okay, So motor is detecting two protective helmets, one jacket, two protective helmets, one jacket,
two protective helmets, one jacket.
Fine.
That's good.
It's the model's jacket into protective helmets.
One jacket.
Okay.
So it's total 337 frames we have.
And it's currently we have process.
The model has processed 177 frames.
Okay.
So 206 frames, 20, 24, 45 to 50 6 to 62, 62 to 70 3 to 95 and to 99.
And it's around over.
So let me download this output demo video and show you what results should we get.
The output is saved in prediction for demo for.
Let me download this output demo video and then we're back with the results.
Well, guys, these are the results from my output demo video three So we can see that the model is
able to detect the protective helmet jackets as well.
So this person is not wearing the jacket.
So the model is not detecting the jacket while these two person are wearing the protective helmet.
So the model is able to detect the protective helmet and this person jacket.
Now, let's test our model on the live webcam.
Although I'm not wearing a protective personal protective equipment, but I want to see if I wear a
mask.
The model is able to detect it or not while on the videos and images.
Our results are very good.
So let's test our model on the live webcam in the next part of the video.
Thanks for watching.