Hello everyone.
In this video tutorial we will see how we can integrate object tracking with YOLO V8 segmentation.
So in this video tutorial I will provide you a step by step guideline, so don't skip any part of the
video and do watch the complete video.
So for this project I will be using this GitHub repo YOLO V8 Segmentation Deepsort object tracking.
So we will be implementing object tracking using Deepsort.
So there are different object tracking algorithms which include sort strong sort nor fair and deep sort.
So as far as my analysis, deep sort algorithm performs best.
So in this project or in this tutorial, we will use Deepsort for the object tracking.
Okay.
So this is the YOLO segmentation Deepsort object tracking repo.
So we will be using this GitHub repo in this tutorial, so I will explain you the complete code step
by step.
In the first step, I have just cloned the GitHub repo which you can do here by clicking on the code.
Just copy this and just paste this over here and just.
And in a simple way you can clone this GitHub repo from the GitHub into your Google CoLab notebook.
So this is the complete CoLab file which we'll use in this tutorial.
First, we will simply run the step.
Then I will give you a detailed explanation of the predict.py file how we incorporate Deepsort algorithm
Deepsort object tracking algorithm into the YOLO V8 segmentation script.
So let's get started in the first step.
Please make sure before running the script that you have selected the runtime as GPU.
After selecting the runtime as GPU, then you need to clone the GitHub repo.
So I'm just cloning this GitHub repo into my Google CoLab notebook over here.
Okay.
So in the next step you can see currently you will see that our current directory is being set that
this project file section.
But as we, we want to set the our current directory as this cloned folder.
So you just need to go here copy path and just paste this path over here.
So now this folder is being set as our current directory.
Okay.
So before we implement YOLO Deepsort Object tracking YOLO Segmentation with Deepsort Object tracking,
we need to install all the required libraries that are required.
Necessary or required to run this script successfully.
So here we have the setup.py file.
This contains all the required libraries.
Or you can say the dependencies that are necessary to run this script successfully.
So to think it's just taking a bit time to open.
Okay, so.
Now I will just install all the dependencies.
So dependencies are basically the libraries that are necessary to run the scripts.
So to install the dependencies, you just need to simply go over here and just run this cell.
It will install all the required libraries that are necessary to run this script.
Okay.
So it might take few seconds, as you can see over here.
So you can see that some libraries are required or preparing requirement already satisfied because some
libraries are by default available in Google CoLab.
While some libraries are not available by default in Google CoLab like classic Gtb, Omega confidence
or other docs material.
So these libraries are not available and they are being installed.
So if you skip this step when you run the training validation or prediction.py script, you will definitely
face the error that the following library is not installed, so it's always better to perform this step
and install all the required libraries.
Or you can say that dependencies that are necessary to run the script successfully.
Okay.
Then in the next step we will go to as we are doing the segmentation, I will go to ultralytics, Yolo
V8.
Then I will go to the segment this folder and set this as current folder.
Okay.
So what here I'm doing is setting the segmentation as my current folder because we are doing YOLO V8
segmentation over here and here we have the training to train our YOLO V8 segmentation model on the
custom data set to validate the YOLO V8 model, a trained YOLO V8 model.
On the custom data set, we use validation bot.py to train validate the YOLO V8 model which is trained
on the custom data set.
Then we have the predict.py file to do the prediction on the pre-trained model on the fine tune YOLO
v8 model, which is being fine tuned or trained on the custom data set.
Okay, so.
Nine dissecting this as my grand folder like grand directory.
Now basically we are implementing object tracking using deepsort.
So we need to download the Deepsort files or upload the Deepsort files to the segmentation folder.
So I'm just downloading the Deepsort files from the Google drive in directly into my Google CoLab notebook.
So here I'm just downloading the Deepsort files from the Google Drive into the Google CoLab notebook.
So now you can see that Deepsort folder over here, but it is in the zip format.
Next we will unzip this folder over here, so this might take a few seconds.
Okay, So very quick.
Okay.
You can see now here we have the Deepsort complete folder over here.
Here we have the all the required files which are required to integrate Deepsort object tracking with
YOLO segmentation.
Okay, so what object tracking is do object tracking assigns a unique ID object tracking.
Simply assign a unique ID to each of the detected object.
This is the purpose of object tracking.
So using object tracking, we assign a unique ID to each of the detected object.
Okay, so now I'm just downloading a sample video from the google drive over here so I can test my YOLO
V8 segmentation model on that sample video.
Okay.
So here I'm just running the YOLO predict.py and YOLO V8 segmentation.
This is the YOLO V8 test segmentation model.
The pre-trained model am just the.
Testing the YOLO VR segmentation model on a demo video which have downloaded from here.
So just run this.
Then we will see the output video and then I will explain you the predict.py file or the predict dot
script in the predict.py file.
And then we can.
Proceed further.
So this might take a few seconds.
Yolo V8X dash segmentation is the most accurate model than other YOLO V8 models because so as we can
say that Yolo V8X segmentation is more accurate, but it is less sparse than other YOLO V8 models,
while Yolo v eight and segmentation is the most fastest model among the YOLO series, but it is less
accurate by YOLO v8 X dash segmentation is more accurate, but it is less faster.
So the update video is being divided into 190 frames and our model process each of the frame one by
one.
So the processing on the complete video is being done over here.
So this might also take some time.
So let's wait and see.
What results do we get?
Okay, so it's about complete.
So our processing on the video is done and our results are being saved.
As in this runs detect.
Okay.
So think YOLO segmentation.
So in this runs detect train and here we have the our results file.
So you just need to go over here.
Simply set this path as this folder where you have the result output video over here and just run this
cell and the output video will be displayed over here in visual over here.
Segmentation with tracking will be implemented.
Okay.
So let me see, let's see what the output video we get.
So this might take few seconds.
So let's just wait for the output video to display over here.
So here is our output video, which we can see in the Google CoLab notebook.
We have displayed the output video into the Google CoLab notebook.
So let me just download this video and show you what results do actually we get.
So just navigate my screen towards this output demo video.
So just give me a minute.
Okay.
So now in front of you you can see the output demo video.
So you can see that we have Now let me just pause this video so you can see that each object or each
person or you can say each object is being assigned a unique ID 82 over here plus.
So in this way we have implemented, uh, or you can say integrated object tracking with YOLO V8 segmentation.
Like you can see the mask over here and the bounding box.
So YOLO V8 segmentation is being implemented.
Plus you can see a unique ID with each of the detected object like 82 with this person 108 with that
traffic light.
Plus you can see these trails as well.
Like you can see that this shows us what which path does the object or the person followed.
So in this way we can see the path the person is following, so in which direction the person is coming
from, which direction the person is coming.
Okay, so in this way we can find the path the person is following.
Okay?
So you can see that using Deepsort object tracking, we are able to assign a unique ID each of the detected
object.
108 92 153 plus.
We have also drawn the draw the trails as well which show the path the object has followed or the person
has followed.
So in a similar way, we can implement object tracking or integrate object tracking with YOLO segmentation.
And we can test your segmentation with Deepsort object tracking on some other demo videos as well.
Let's test on this demo video over here.
Okay, so I'll just run this script and see what results do we get from this output video.
So I'm just running the Yolo v eight Segmentation.
Segmentation model on the S3 dot mp4 video.
This is the Pre-trained model.
You can also take the custom model and am just running the predict.py file and am its processing on
the video so it has divided the complete video into 508 frames.
You can see here and it's doing the processing on each of the frame one by one you can see over here
and.
So it's a two processing on 24 frames to 97 frames are done out of 588 frames.
Okay.
So here we are just testing a segmentation model on this demo video, which is test 3.4.
Okay.
So the processing on the video is almost done.
I have already run this script previously as well, so it will not.
So that's why you can see the output video over here as well.
So our output video is saved in run detect train to test three dot MP4 is the name of our output video.
Okay, so here is our output video.
Let me just download it and show you what results do we get over here if I just run this cell again?
So it might take very much time.
So just to save the time, I'm just showing you the output video.
So now you can see the output video in front of your screen.
Let me just pause this video so you can see that each of detected object, like the car over here is
being assigned a unique ID, one, ten, 11, six, eight.
Plus we have this trails as well, which shows that the path object is following.
So you can see this trails that you get from this trails, you can see that the car is coming in from
this path and it's going straight.
Okay.
If the if it's following this, this, this, this, then the trails will be like this.
Okay.
So the trails show that what path the object is following.
Plus we have the unique ID with the detected object.
So in this way we have integrated the sort object tracking with YOLO V8 segmentation.
Okay, so let's go back towards the code.
If you want to run this complete video, you can see that it's done on each of the object, plus you
can also see that.
At the same detect objects like car have the same color of the bounding box as well as the mask.
While different objects like truck as the different color of the bounding box as well as the mask.
Okay, so you can see that here we have the directory red truck, although it's separated wrongly.
It's not a truck, it's a car.
But you can see here, the color of the mask is different.
And color of the mask and bounding box are different than the car.
Okay.
For the car, we have the pink color of the and orange color of the mask.
While you can see over here in the case of truck, we have the brown color of the bounding box and the
brown color of the mask as well.
So each object.
So the object of the same same objects have the same color of the mosque and the same color of the bounding
box, while different objects have the different color of the mosque and a different bounding box.
So let's go back to our code and spend the rest of the code as well.
So if you go back to the code and let me open the predict.py file over here.
Okay.
So in this predict.py file I have integrated the object deepsort object tracking code with the YOLO
segmentation.
Okay.
So here you can see that i have just imported that deepsort from the deepsort pytorch folder.
I have imported a deepsort module over here.
You can see that Deepsort module is being imported from the Deepsort PyTorch folder over here.
And then I am.
Boring to get congregation files from the utils parser over here.
So I'm just from this.py file.
I'm quoting the get dash get dash configuration module.
Okay.
And here I'm just importing the deepsort module because we are implementing object tracking using deepsort.
Okay.
So here I've initialized the tracker deepsort object tracker over here you can see.
Okay, so here we are, just converting the output received from the YOLO segmentation model into a
format that is compatible with the YOLO V8 or that is compatible with the Deepsort object tracking.
So here what we are doing is that we are converting the LB coordinates or you can see the output of
the YOLO V8 segmentation model into a format which is compatible with Deepsort object tracking.
So the output of the YOLO V8 segmentation model is being converted into a format that is compatible
with Deepsort object tracking.
Okay, so what basically we are doing is that.
If.
Open the paint file over here.
Let me just.
Give me a minute.
Okay.
So let me just navigate my screen over here.
Okay, So now you can see this file.
Let me just increase this file size.
Okay.
So you can see that if this is the bounding box output received from the YOLO V8 model.
YOLO, V8 segmentation model.
So in the YOLO V8 segmentation model, let me just write.
We have this form of x.
Y1Y.
This is the this and this coordinate.
And this is AX2Y2 coordinate.
Okay, so what does YOLO V8 segmentation model does it.
It converts these basically through YOLO V8 segmentation model.
We get the output in the form of top left corner and the bottom right corner.
So we get these four points.
The first point is x one.
The second point is Y one, the third is x two and the fourth is Y.
So using YOLO V8 segmentation model, we get these four point as the output, but object YOLO.
Deepsort object tracking requires the output in the form of XY.
In Deepsort object tracking, we require the center coordinate.
Okay.
What is the center coordinate of the bounding box?
Okay.
Plus width and height of the bounding box.
This is the height and this is the width of the bounding box.
So in Deepsort object tracking, we required the center coordinate of the bounding box.
Okay.
And the width and the height of the bounding box.
Okay.
So.
Using this function.
Over here we convert the output received from the YOLO V8 model.
YOLO segmentation model into a format that is compatible with the deep sword object tracking.
Like we convert the x1, y1 x2 y2 coordinates into x, y, z center coordinates of the bounding box
and the width and the height of the bounding box x, y, z.
Present the center coordinates of the bounding box and the represent the height, width and height of
the bounding box.
So we are not using this function over here.
Plus we are.
What we are saying is that if the label is a person, then assign the bounding box mass and the label
this color.
If the label is cat, then assign the bounding box mass this color.
So we are just defining the color and.
And here we have created the Xbox One and Xbox function is used to create a bounding box around the
detecting object.
So using Cv2 dot rectangle, I'm creating a bounding box around the detected object and using this dot
rectangle, I'm just creating a rectangle above the bounding box so I can put the label text over here.
Like this is the person, this is the car over here.
Okay, now we have the draw boxes function in draw dash boxes function we call the dash box function.
Okay.
And we also implement a deep sword object tracking in this function as well.
Okay.
So you can see that here we are assigning a unique ID to each of the detected object over here.
Okay.
So in data decoded in this data.id, that attitude ID is basically and this is a dictionary in data
you we are just.
Sorting the center coordinates of the bounding box for center coordinates of the bottom of the bounding
box over here.
And then I'm just calling the box function so that I can draw the rectangle around the so I can draw
the bounding box around the object and assign or also draw the rectangle to add the label as well.
Okay.
Then using this function using cv2 dot line, I am drawing that trails.
Okay.
So I have already told you what are the trails?
So these are the trails.
The pink line.
You can see over here the pink line.
You can see over here.
These are the trails.
So using cv2 dot line, I'm just drawing that trails over here as well.
Okay, then here I'm just incrementing the segmentation, creating a mask around the detected object.
Okay.
And here I'm just incrementing the deepsort with the object tracking as well.
Okay.
And I'm just calling the draw dash boxes function created above to implement the segmentation with deepsort
object tracking.
YOLO segmentation with Deepsort object tracking.
So that's all from this video tutorial.
See you all in the next video tutorial.
Till then, bye bye.