Hello everyone.
In this video tutorial we will see how we can integrate object tracking with YOLO v eight segmentation
to do vehicles counting.
You can see over here and the speed estimation as well.
So this is a complete end to end video tutorial in which we will learn how we can integrate object tracking
with YOLO, V8 segmentation and also how we can do vehicles counting with speed estimation.
So don't skip any part of the video and do watch the complete video and let's get started.
But this is the corner file which I will use in this video tutorial.
As in this video tutorial, we will see how we can use YOLO V8 segmentation with object tracking to
do vehicles counting and speed estimation.
So let's get started.
And the first step, I will explain you the complete code lab code.
Or we can simply say that in the first step I will run this complete code lab script.
So after executing the complete code lab script and showing the output demo video, then I will explain
the complete code in the predict.py file and how you can implement this project on your side.
And the complete understanding of the port will be provided in this video tutorial.
So for this project I will be using this GitHub repo yolo VR segmentation with deep sword object tracking.
I have created this GitHub repo and it's available publicly on GitHub.
So in this GitHub repo we will incorporate the vehicles counting and speed estimation port.
So currently we have the YOLO V8 segmentation with Deepsort object tracking GitHub repo available.
So currently object tracking is being incorporated in the YOLO vs 88 segmentation module.
What next?
We need to do is we need to add vehicles counting and speed estimation code in this already available
code to make it work and able to do the vehicles counting and speed estimation as well.
Okay.
So here I'm just closing this GitHub repo over here.
So by just going over here, copy it and just pasting this path over here.
In the first step, we will clone this GitHub repo into the Google CoLab notebook.
So we will able to assess all the files of this GitHub repo into this Google CoLab notebook.
So just run this cell.
So it will go on the GitHub repo and you will be able to see the clone GitHub repo folder over here.
So now we have the segmentation Deepsort object tracking folder over here.
So now if we see our current directory, our current directory is being set to this project file section.
We need to set our current directory to this folder which we have cloned.
So just copy the path and just paste this over here.
So in the next step, what we need to do, we need to install all the required libraries that are necessary
to run this project.
So if you skip this project, if you skip this step, some libraries might not be installed.
And if you run the training validation or predict.py script, you might face the error that the following
library is not installed or the following library like Hydra library is not install Matplotlib library
is not installed or the Seaborn library is not installed, so it's necessary to install all the required
libraries before, uh, before running the training validation or prediction.py script.
So to install all the required libraries just run this cell so it will install all the required libraries
that are necessary to make this script successfully run.
Okay.
So now as we are doing segmentation, so we'll just go to ultralytics YOLO.
Then I will go to YOLO, then I will go to V8, then I will go to segment and I will select this folder
as my current directory over here.
Okay, so this folder is being set to my current directory.
Now basically we are integrating object tracking with YOLO segmentation.
So to integrate object tracking with YOLO segmentation, we need to have the object tracking files available
as well.
So to implement object tracking, I am using deepsort the state of the art multi multiple object tracking
algorithm.
It is.
Okay.
So in the next in this step I am just downloading the deepsort object tracking files from the google
drive into the our Google CoLab notebook.
So as we are implementing object tracking using Deepsort.
So here I will install or you can say that download the Deepsort files from the Google drive into my
Google CoLab notebook.
So in this way we can integrate the Deepsort object tracking script into our YOLO segmentation.
So now I will zip this file.
So now you can see here we have the Deepsort PyTorch folder over here as we are implementing object
tracking using Deepsort.
So next we need to download a sample video for testing from the Google Drive.
Okay.
So to run the YOLO segmentation model with object tracking, I am just downloading a sample video from
the Google Drive.
So now I'm just downloading the by default.
Here is a predict.py script which will do the object segmentation with object tracking.
But we need to do the vehicles counting and the speed estimation as well.
So I will download the updated predict.py file in which I have cooperated or integrating the vehicles
counting port as well.
Okay.
So I'm just downloading the updated predict.py file.
So.
In this way we can implement the vehicles counting and the speed estimation as well.
Okay.
So that's John's this.
So this might take a few seconds.
And in the end you will get this output demo video here as well.
So we will test on multiple demo videos, which you can see over here.
So let's do it one by one.
Okay.
So basically now I'm running the python predict.py file on a demo video.
Test1 dot mp4 in this.py file.
I have integrated the speed estimation code as well into the object tracking and.
You got a weird segmentation code.
So in the segmentation and object tracking code, I am integrating or adding the speed estimation and
vehicle counting code as well.
Okay.
So vehicle counting and speed estimation.
I think we should write over here.
Vehicle counting and speed estimation.
Okay.
So now I'm just adding the vehicles counting and speed estimation code in the predict.py file.
So predict.py file contained in the predict.py file.
We have integrated object tracking with your segmentation.
Thus we have also added the vehicles counting and speed estimation code.
So when run on this demo video it divides the complete video into 943 frames and it process each of
the frame one by one and it tells the predictions in each of the frame like one person per truck.
So each video frame is each frame in the complete video is divided into 943 frames, and each frame
processing on each of the frame is done.
And here are the results of the processing on each of the frame.
Like 11 cars, one truck.
So the end up using this yellow segment dash segmentation model.
We are doing the processing on this complete video frame by frame.
Okay.
So as this run and we have the output demo video, I will explain this complete code video to you as
well.
So I'm just waiting for this to run.
I have already run the script for this video tutorial so you can see the output demo video over here
as well.
So just to explain you the whole thing, I've just running this script again here, so this might take
few minutes or some more time.
Okay.
LucasArts are running and will complete its.
Okay.
So it's 790 frames, 803 frames, 809 frames.
So what basically we are doing is that let me explain you.
We are integrating object tracking with YOLO segmentation and then we are using object tracking with
YOLO segmentation to add the vehicles counting and speed estimation.
Or you can say to implement the vehicles counting and speed estimation.
Okay, so we have run this script and our output is saved in run train folder.
So if we go to the over here runs detect train folder, here is our output video test one dot mp4.
So I've already displayed this output video over here, so I am not running this cell over here, but
it will take time.
So I've already displayed this output video over here, so we just need to go here and copy path and
just paste this over here.
And when we run this cell, this output video will be displayed over here.
So let me download this video and show you what results do actually we get.
Okay.
So let me just play this video and let me just navigate my screen towards this output video.
Okay.
So you can see over here we have the.
Or because counting you can see over here the vehicles going in the north direction are the vehicles
entering and the vehicles going in the south direction.
Are the vehicles leaving?
Okay.
Plus, you can also see over here.
Okay.
So let me just go here.
Let me do it a bit more.
Okay.
So let me just stop it over here.
So now you can see that we have the vehicles counting and speed estimation as demanded as well.
And we can see the speed estimation over here.
Plus, next, we have the vehicles counting code over here as well.
So because going into this direction, are the vehicles entering and the vehicles going in this direction,
are the vehicles leaving?
So we can simply say that the vehicles going in the north direction are the vehicles entering and the
vehicles going in the south direction?
Are the vehicles leaving?
Plus, we can have also integrated object tracking with YOLO V8 segmentation.
So you can see that we have assigned a unique ID to each of the detected object like 145 164 153 139
137 plus we have also drawn the trails as well.
Like you can see these trails.
So these trails basically show that what path the object has followed.
So you can we can say that the object is following this path, like coming from here and just going
over here.
So now what we have done is we have assigned a unique ID to each of the detected object, plus we have
done the speed estimation as well, Plus we have done the vehicles counting.
We have the total count and the sub count of the cars truck.
And here we have also the total count 16 and the sub count of cars and trucks as well.
So if we play this video from start, you can see that when the vehicle crosses this line.
We have the account crosses this line.
We have the vehicles entering count.
And when the vehicles crosses this line, we have the vehicles leaving count.
Okay.
You can see over here.
Okay, That's cool.
So it's working quite fine.
Let me explain you one thing over here.
Okay.
I'm just trying to post a video, so let me just explain this one thing over here.
Okay.
I'm just looking for a perfect example.
Just.
Just give me a minute.
Okay.
So that is the what I'm trying to explain.
So you can see that these two other trails, when the trails intersect with this green line, which
have a manually drawn, we can see the increment in the account, like the total count.
And if it is a car, we can see the increment in the cars count as well.
So what we are doing is that when these trails intersect with this white line, then we see the increment
in the total count and in the cars count as well.
Okay, So important thing is that like when this trails like this brown line over here, when this trail
will intersect with slight green line, then we can see the increment in the count of the vehicles leaving
and the sub count.
If it is a truck, we will see the increment the truck count as well.
Okay.
So this is currently a green line.
When the these trails intersect with this line, this green line turns white.
Okay.
So you can see that originally it's a green line, but when the trails intersect with this line, this
line turns white.
Okay, So I'm just checking if there is any other pattern.
Okay.
So that you can see that currently a green light when this trail intersects with this line, this turns
into white.
Okay.
So similarly, if the trail is coming from here, intersect with this line, this line will turn white
and we will see the increment in the total count of the vehicles leaving.
And here is the sub count as well.
Okay.
So in the similar way you can test it on another demo videos as well, have just test it on this output
demo video as well.
And we can see the count over here as well.
Okay.
So.
Okay.
So this is all.
Let me explain you the predict.py file and then we can end this video tutorial as well.
So in the predict.py file I have integrated object tracking with YOLO V8 segmentation plus I have added
the vehicles counting and the speed estimation code in the predict.py file as well over here in the
first step as implementing object tracking using deepsort.
So I'm just importing the deepsort modules in this.
Predict Pyscript.
Okay.
And here I'm just initializing the Deepsort tracker.
Then here I've created two dictionaries object counter one and object counter in the object counter
one will store the count of the vehicles entering and the object counter will store the count of the
vehicles leaving.
And in the count up I will store the total count of the vehicles entering and the count down.
I will store the total count of the vehicles leaving and in the object counter one dictionary will store
the sub count like the cars, trucks and buses, count of the vehicles entering and the object counter
will store the sub count of the cars, trucks and buses leaving.
Okay.
And here I've initialized the Deepsort tracker over here and here I'm just converting the output received
from the YOLO V8 segmentation into a format that is compatible with YOLO V8 and visual that is compatible
with Deepsort object tracking.
So here I'm converting the output to see from the YOLO V8 segmentation to a format that is compatible
with Deepsort object tracking and don't need this function currently in this project.
I have also explained the complete color from labels in the previous videos and here I'm just creating
a checkbox function to draw the rectangles around the detected object or the bounding boxes around the
detected object.
You can see that I'm using Cv2 dot rectangle over here.
I'm drawing the bounding boxes around the detected object and using this cv2 dot rectangle, I'm creating
a rectangle above the bounding box to put the text of the label and the unique ID as well.
And here I've just created a function estimates period to do the speed estimation in to calculate the
distance in pixels per stamp, calculating the distance in pixels to calculate the distance in pixel.
I'm using the gradient distance partner.
This is the gradient distance formula.
After calculating the distance pixel, I am creating the distance in meter by dividing the distance
pixel over pixel per meter.
And here I've set the pixel parameter value as eight so you can make it dynamic as well.
Closer to camera one and away from camera 20.
So this way you can set it dynamic and here we have the time constraint.
This is a constraint.
You can adjust this time constraint as per the to get or you can fine tune this time to set to get more
better results.
And if you go here speed formula.
Okay.
So basically we have speed is equal to distance divided by time, but here we are doing distance multiply
time constraint because this is a frequency, this is not time.
So if we see that frequency is equal to one by time, okay, one by one divided by time.
So here we are doing speed is equal to distance by multiply by frequency.
Okay?
And here we are getting the speed and to find the intersection between the trails and the manually drawn
line, we are just creating the function intersect over here and here we have the function get track
direction to calculate the vehicles in moving in the north direction.
Then we will do the vehicles count of vehicles entering.
If the vehicles is moving in the south direction, then we will do the count of the vehicles leaving.
Okay.
And will be just calling this function estimate speed over here to calculate the speed.
And here I'm just calling the function get direction to calculate the direction if the vehicles are
moving in the north direction or the south direction.
Over here I am just moving a bit rapidly because in the YOLO object tracking I have already explained
this code step by step, so I'm just moving fast.
If you don't understand the sport, just go back to the YOLO V8, the Deepsort object tracking lecture
and to understand this code.
That's why I'm just moving fast over here in the YOLO V8 segmentation with Deepsort object tracking.
Okay.
And if they here we are doing the vehicles, count of the vehicles going the south direction, which
is the vehicles leaving, leaving.
And here we are doing the count of the vehicles going in the north direction, which is the vehicles
entering over here.
And I am just displaying the count of vehicles entering and the count of vehicles leaving in the UI.
So that's all from this video tutorial.
I hope you have learned from this video tutorial.
See you all in the next video tutorial.
Till then, bye bye.