In this video tutorial, we will learn how we can create each of the detected object using Deepsort
object tracking algorithm.
We will be integrating Yolo v9 with Deepsort object tracking algorithm, so we will use YOLO v9 to detect
objects in image video or in a live webcam feed.
And then we will be using Deepsort object tracking algorithm to track each of the detected object until
that object remains in the frame of a video or in a live webcam feed.
So what we do in object tracking?
In object tracking, we basically assign a unique ID to each of the detected object.
And then using that unique ID, we track that object until that object remains enough.
Uh, in remains in, uh, in the frame of a video or in a live webcam feed.
So using that unique ID that we assigned to each of the detected object, we then track the object.
We can also draw trails, follow the path that check the path the object is following.
So that's all we are going to learn.
In this tutorial we will be integrating YOLO v9 with Deepsort object tracking algorithm.
You all know your V9 was released last month on 21st February 2024, which is the state of the art object
detection algorithm and YOLO be nine outperforms all the previous object detection algorithms as well.
While Deepsort algorithm is the state of the art object tracking algorithm.
So by combining the state of the art object track detection algorithm with object tracking algorithm,
we will be doing object detection last object tracking.
So that's what we are going to learn.
In this tutorial.
Thus, we will also see if the object gets blurred or hidden.
We will also be tracking that object and that object unique ID will remain the same.
Okay, so let's get started.
So over here you can see in my jump I have opened PyCharm over here and I have Python 3.10 installed.
You can see over here.
Plus I have just created a project over here by the name of init Deepsort.
So I have created this project in my app directory.
So in the same way you can create this any project in any of your directory.
Plus you can also use Visual Studio.
I'm using PyCharm.
You can also use Visual Studio as well.
Between the first step, we'll go step by step.
So if you just see this note while I've written all the steps, so which I will be pointing.
So first we need to clone the repo.
So I will just open the browser over here.
It.
And I will just go over here and.
I will just write your lovin line GitHub over here.
Lingua.
Volume nine.
Get up.
So you'll just search for this guadalmina and get up over here.
And I will just open this post link from here.
Okay, so this is the Yolo v9 official GitHub repo.
It has around 7000 stars.
So I will just, uh, clone this complete, uh, repo into my, uh, PyCharm directory.
So I will just click copy this link from here.
Then I will just go to the PyCharm over here.
And then I will just here.
And then I will just write git clone.
And here I will just add the repo link.
So now it will clone this complete repo into my local directory over here.
And you will find this okay.
So it's done now.
So if I just refresh this so you can find this complete Golovnin repo over here.
So now in the step number two I will just start this uh yolo benign repo, uh, which I have cloned
as my current directory.
So I will write CD yolo VI over here okay.
Bernard.
This is being.
So now if I just open the requirements.txt file, the requirements.txt file you can find, I have the,
they have listed all the required packages that are required uh, to run the repo successfully.
Okay.
So okay.
So I will just uh, as we will be integrating object tracking using Deepsort algorithm.
So we need to add few more uh, packages as well.
So which is you can add it and plus I will see if the numpy package is over here.
So okay so I will require numpy package.
Less than.
1.24.
And so we will be getting an error.
So it's always factored.
So this and we also required g down package because we will be uh installing demo videos or deepsort
files from the drive into the tracker into this PyCharm.
Uh ID okay.
So now in the step number two, what is install all the required packages okay.
So we will be installing all the required packages over here.
Well, now to install all these required packages I will simply write over here.
Pip install minus r requirements dot txt.
So we don't need to install all these packages one by one if we just write requirements dot txt.
Pip install.
It will install all the packages that are required listed in this requirements.txt file.
And I will now click on enter.
So now all these packages will be get installed over here.
So this will take some time, but all these packages to get installed.
So let's wait for these packages to get installed and then we will go ahead.
Now you can see that the packages are installed.
So now in the step number three, what we need to do is we need to download a Deepsort files.
So I will just copy this link from here.
Yes.
Copy this.
Okay.
And I will just paste this link over here and I just click on enter.
So this will download the Deepsort files from the drive into this Google Colab notebook or into this
PyCharm ID.
So I've already placed this deepsort zip file into my drive, and I'm directly downloading the Deepsort
PyTorch zip file, uh, from my drive into this PyCharm IDE.
Uh, so now we just need to unzip this deepsort files.
So what I will do for this, I will just, uh, click over here and create a new script.py file.
Uh, in the script or by file.
I will just write some code, uh, to zip, uh, unzip these deepsort files.
So over here.
Import.
I will just write over here import zip file.
With zip file.
Uh.
Not zip file.
Here I will just write the name which is deepsort by dot dot zip.
But we need to read this.
As the dash ref.
And we need to extract all these complete files.
So just do this.
And over here I will just run this script file.
And let's see how does it works.
Okay.
So do we have the okay.
So now you can see I've just, uh, completely unzipped this file over here.
Okay.
So there is a function in Python unzip.
But that function does not work in PyCharm.
I have tried that function in Google Colab.
It works fine there, but that doesn't work in PyCharm.
So in the deepsort PyTorch, uh, if I have not unzipped it.
So you can see here we have a Deepsort folder where we have the uh, complete deepsort.py file code
as well.
And we have the configuration and readme file over here as well.
So this is what we all require okay.
Next what we need to do is we need to download a demo video from here.
So I've already added the link over here.
So okay so then we are just uh good as well.
So now in the terminal I will just write add the link over here.
So this will download a demo video from a drive into this PyCharm IDE over here.
Okay.
So now you can see we have the demo video over here as well.
So if I just re okay so no need to.
So you can see here test dot mp4.
Uh we have this demo video over here as well on which we will be doing detection and tracking okay.
So in next part I will do is um, so in the detect.py we have all the object detection code.
So I will just add another file detect dash du that tracking.py in which we will be doing object detection
plus tracking over here.
Over here you can see that I have added detect-dual-tracking.py file.
So here I will be uh, adding the deepsort code.
I have added some deepsort code over here which include initialization initialization of Deepsort algorithm.
So over here what I have done for now is I have initialized or I have imported all the Deepsort required
packages over here, which you can see over here.
And then I have initialized Deepsort over here.
I have initialized the Deepsort tracker over here, and all the parameters which you can find over here.
So the important parameter we should, which you should need to know.
I have uh, written all the comments about each of the parameter above.
You can read that as well.
So here is the maximum age parameter.
So you should uh need to know about that parameter.
So in the maximum age uh what we define is that for example uh due to an occlusion or if uh, for any,
in any case, if an object tracking ID is lost for an object, um, so in the maximum age we define
how many frames it should wait to get that, uh, ID back.
So, uh, for example, if there is a car and we have lost that, uh, car object tracking ID due to
an illusion, so how many frames that it should wait for it to get back?
Or if it doesn't get back in this much frames, then it will assign a new ID.
So for example, if I set the maximum age value to 20.
So it will wait 20 frames, uh, to get the object tracking ID back, and if in 20 frames it does not
get the object tracking ID back for that specific object, it will assign that object a new ID.
So if you can see over here if an object tracking ID is lost, this parameter determines how many frames
the tracker should wait.
Wait before assigning a new ID so you can see all the details over I have written over here.
Then over here I have to define the minimum confidence, maximum distance and the RMS maximum overlap
maximum IOU distance.
So you can read all those comments.
I define every parameter over here.
This will take quite some time so I will skip this.
And here I've just created a function initialized Deepsort.
So I have just uh initialized that over here.
Okay.
And here I have uh, because our YOLO v nine model has been trained on the Coco data set.
So the Coco dataset consists of 80 different classes.
So I have written all those 80 different classes names over here.
And then uh, for example here I've defined the color label.
So if we have the if the if in a video frame or in a live webcam feed, uh, person is detected, then
we have assigned, uh, if the person is detected in a live webcam feed or in a video, then the bounding
boxes will have the following color.
So if the car is detected, then the bounding boxes will have the following color.
If the motorbike is detected in a video frame image or in a live webcam feed, then the bounding boxes
will have the following color.
And if the box bus is detected in a video frame or in a live webcam feed, then the bounding boxes will
have this color.
Okay.
And over here, uh, they we have loaded the model.
We have applied non-expression over here.
So this is all what is going over here?
So from now where I will just add the object tracking code.
Let me show you.
So from here I will start adding the object tracking code over here.
But before I add this so you can see over here okay.
So so in the YOLO v nine we get the output in this form.
Like you can see that uh, the Yolo v nine object detection algorithm returns the output.
In this form we have the bounding box coordinates.
We have the uh top right corner uh top left corner bounding box coordinate.
You can see this is the top left corner and this is the bottom right corner.
So the in the yolo uh, the yolo v nine uh, model outputs return the top left corner bounding box coordinate
and the bottom right corner bounding box coordinates.
So this is what yolo v nine returns top left corner bounding box coordinates and the bottom bottom right
corner bounding box coordinates.
So we need to pass the Yolo v nine object detection algorithm output to the Deepsort algorithm.
So the Deepsort object tracking algorithm, uh, accepts uh the input in the form of width height of
the bounding box.
So this is the width okay.
Here I'm wrong.
This is the height.
So this is wrong if I just, uh, correct this up over here.
Okay.
So this is.
Uh, so let me just correct this.
So this is basically.
The height, and this is the width of the bounding box.
So the Deepsort object tracking algorithm accepts input in the form of height, width of the bounding
box and the center coordinates of the bounding box.
So these are the four parameters that are required, uh, that we should pass in.
The input of the Deepsort object tracking algorithm is the center coordinates of the bounding boxes,
and the width and the height, width and the height of the bounding boxes.
So from the Yolo v9 object detection algorithm, we get the uh top left corner, uh top left corner
bounding box coordinates, and the bottom right corner bounding box coordinates.
So this is what we get from the Yolo v9 output.
So we need to transform that output into the height and width of the bounding boxes and the center coordinates
of the bounding boxes.
So first we need to do this.
We need to transform the benign output into this height width and the center coordinates of the bounding
boxes.
So over here I will start writing the code.
So from, um, nine, we get that, uh.
Top left corner bounding box and the bottom right corner bounding box coordinates, which we are here.
Thus, we also get the confidence score and the class names.
So these are the output we get from the YOLO v9.
So here you can just write.
So X1Y1 represents the top left corner.
And X2Y2 represents the bottom right corner.
Bounding box coordinates.
So the output in the form of tensors.
We need to convert the output into integers first.
So I will just write over here int x one, int y one and x two and int y two.
So we need to have transformed the output from tensors into integers.
So now we will find the.
Enter coordinates.
For each of the.
Detected object.
So, as I told you, that in Deepsort algorithm, uh, we need to pass the input in the form of center
coordinates for each of the bounding box of the detected object and the width and height of the bounding
boxes.
Okay.
So we will find the center coordinates first.
And then we'll find the width and the height of the bounding boxes as well.
So I will just ride over here.
So here I will just find the width.
Eye of the.
Only.
So to find the weight.
So you can say that, uh, basically we here we have x1.
Okay.
So if I just write over here.
And.
And X1.
And here we have.
Uh x2.
So if we subtract x one from x two, we get the width.
And if we subtract y one from y two, we get the height.
So I will just I will just write over here x one minus x four.
So this will give me the weight.
And for the height I will just write over here.
For the Deepsort algorithm.
Accept input in the form of.
Center coordinates and width and height of the bounding box.
We'll be right over here.
We're.
Okay.
So now we need to append all these width.
And how?
Because in a frame there will be multiple objects that will be detected.
And so we have uh center coordinates for their multiple objects and width and height or values for multiple
objects that are detected in our video frame in image or in live webcam feed.
So I will just here append dot append.
Okay.
Then I will also append all the confidence values over here as well.
Plus we have also have these class, uh, IDs as well.
For example, the class ID for the person class is zero.
So but these values in the form of tensors.
So we need to transform those values from tensors into integers as well.
And I will append these values over here.
And, uh.
Okay, so that's done.
Then we have.
So now we need to convert these values into tensors again.
And as I told you, that a deepsort algorithm accept, uh, input in the form of, uh, Deepsort algorithm.
Object tracking algorithm in accepts input in the form of, uh, center coordinates and width and height
of the bounding boxes.
And along with the confidence score and the class names.
Uh, content store and the class IDs.
We don't have the class names yet, so.
So as we have transformed our output in the form of tensors and then have confidence score over here
as well.
So here we have the class IDs.
And is IMS represents the current frame in which we will be doing this.
So if we have deductions like if we have some deductions for.
Then we will be doing the tracking as well.
So.
So now you can see over here, uh, in the Deepsort object tracking algorithm, our input, we have
passed the centre coordinates, width and height of the bounding boxes and the confidence score and
the class IDs and the current frame.
So the what output we get from the Deepsort algorithm.
So here in the Deepsort algorithm.
So if I just write.
What year?
Deepsort algorithm.
Output.
So.
So this is what we get.
If I just again zoom it up for the.
Okay, so here we can.
So in the Deepsort algorithm output, we get a.
Uh, the bounding top, right?
Uh, uh, now, this is a top left corner bounding box coordinate, and, uh, the bottom right, uh,
bounding box coordinate.
So Deepsort algorithm output returns the top left corner bounding box coordinate and the bottom right
corner bounding box coordinates, which you can find over here.
So we get the bounding box coordinates in the Deepsort algorithm.
Output a unique ID for each of the detected objects, plus the Deepsort algorithm.
Also return the class ID.
For example, for the person class, it's zero.
So there are three things that we get from the Deepsort algorithm output.
One is the bounding box coordinates.
The second thing is the unique ID for each of the detected object.
This is what we do in object tracking.
In object tracking, we assign a unique ID to each of the detected object, and we also get the class
ID for each of the class.
For example, for person class, the class ID will be zero.
So in the bounding box coordinate we get the.
Top left corner bounding box coordinates and the bottom right corner bounding box coordinates.
Okay.
So.
First thing will I will write over here.
First we get the bounding box coordinate.
We get the top left corner bounding box coordinates and the bounding bottom right corner bounding box
coordinates.
Unless we get the IDs for unique IDs for each of the detected object.
Plus we also get the object ID, for example, for the person class, the object ID zero, and for the
car class it's 1 or 2.
Okay, so now we need to draw those bounding boxes around each of the detected object.
And we need to add the ID uh for this as well.
So I will be creating a function by the name draw dash boxes.
And.
In the drawer dash boxes, as I will be writing the code so that I can draw bounding boxes around each
of the detected object.
Assign a unique ID for each of the object, and write the class name above each of the detected objects.
So.
So in the input I will pass in the bounding box coordinates.
The.
Then I will be, uh, drawing the trails.
So to see what path the object is following.
So to track the path.
Then I will be passing a unique ID for each detected object in the input.
And, uh, object ID, which is the class ID as well.
So the object is basically the class ID like for the person class is zero.
So these will be all the inputs.
So let's write that door.
Dash box is function.
Okay, so you can just get around this.
So here I will be creating a writing down the code for the function draw dash boxes.
The input will be the frame.
The bounding box coordinates.
Dot rail so that we can track the path.
The identities.
So we can initially set these values to none over here.
Then we have the categories as well, so we can set this as none as well.
So categories are basically the class IDs.
Then we have the offset.
We can set this to zero.
So let's try down the code.
Complete code over here.
So let's find out the height and width of the current frame.
Okay.
Then I will write.
Okay.
For a common box.
So now we will look through each of the bounding box coordinates.
Okay.
Oh no.
I'm looping through each of the bounding box coordinates over here.
So.
Um, in the bounding box coordinates.
Let's add some offset value over here.
And.
So here we have defined the offset value over here.
So so now we are looping through each of uh all the uh bounding boxes.
And here we are just getting the coordinates values for each of the bounding boxes.
So we are looking through each of the bounding box coordinates one by one.
For now, we can find the center point of the bounding box.
So find the center point.
The.
So here we are, just finding the center point of the bounding box.
Then over here.
We are calculating the class IDs for each of the detected object.
Then, uh, using the class IDs.
For example, if the there is a person, then we will have the bounding boxes will have this color.
So we will simply write over here.
And here we will pass the class ID.
Then for each of the detected object, we will have a unique ID for each detected object so that we
can track that object using that ID.
So, uh, in that entities, we have the unique ID for each of the detected object.
Um.
So now we will just, uh, draw a rectangle around each of the detected object.
Or you can simply say we are drawing bounding boxes around each of the detected objects.
And the thickness of the bounding boxes will be two.
So now what we will doing is that we will be using that ID so that if the ID is zero, then the there
will be person class.
If the ID is one, then bicycle.
If the ID two is then it belongs to the car class.
So.
So in the category, we have the ID for each of the class.
And using that ID will find the class name.
So in the label, what we will display is that, uh, first I will display the ID.
Then we have the class name over there as well.
But so do things we have in our, uh, uh, above the bounding boxes where we will display the ID unique
ID, and then we have the class name.
We will not be displaying the confidence score over there.
So now we're finding the text size.
Or we want to team up on that 0.5.
Now we will be creating a rectangle above the bounding boxes as well.
And we want the rectangle to be filled, so we'll just write minus one.
And then we will be adding the text in the rectangle which we have created above the bounding box.
In the text, we will be displaying the label which contains the class name and the uh with uh id unique
ID of that object.
Just setting the location of that label.
I mean, the color will be.
White for the text.
And the thickness of the text will be one.
So now we will be creating a circle at the center of each of the object.
Okay, so now we have run this code.
Plus we also want to draw the trails over here as well.
So we can track the path the object is following.
So I will just write down the code for this over here as well.
Okay.
Right.
If I lean not in data deck.
So data deck is a type of dictionary that we have initialized above.
And it has a maximum length of.
64.
Okay.
So now we'll just write over here if Broad Trail.
So if the user wants to draw the trails.
For I in range.
Is none.
Oh, no.
We'll just get greater thickness over here.
So we need to initialize the numpy package at the start over here as well.
So I will just write import numpy as np.
Okay okay.
If okay.
So we have already imported this.
So no need to export that.
Not imported.
Or.
We need to multiply this by 0.5 over here.
Or we can make it 1.5.
Next we need to draw the trail.
So to draw the trails we will be using CV two dot line so that we can draw trails.
The input will reframe.
Okay.
One.
And here we have data dq ID.
One.
Then we have the color and the thickness as our input as well.
And.
So that's quite fine for now.
I think I have written all this, so let me execute this script file and show you how our output looks
like.
Before I run this, I forgot that I have not downloaded, uh, the model weights.
So I will just go over here and just click over here and the model weights will start downloading.
So if I just go over here let me just close these files.
And.
So I should have over here the model which saved.
So if I just check over here.
Okay.
Uh, so let's wait for this model.
Weights to get downloaded.
So it will take a few more seconds for this model.
Weights to get downloaded.
Okay.
So they are about to complete okay.
So that's done.
Okay, so let me just add over here.
So I will just add there.
Over here.
Now you can see it.
Okay, so let me just run this, uh, now in the PyCharm.
So let me just run the script and show you how our output looks like.
Now to run this up over here, we just need to write this Python detect-dual-tracking.py.
Here we I will just define the weights file which we have downloaded.
And that's my input video.
And I'm just running it on CPU because I don't have GPU in my local computer.
So CPU and dash video image so that uh, it can show me the output image or output in the it will open
a window where I can see the output video over here as well.
And my output will be saved into this, uh, folder as well.
So that's all this upload.
Okay.
No such file order.
Okay, so I think mistake is on my side.
Let me.
So we need to set the inline folders current directory.
So now let's run this up.
A.
So this will take few seconds before it starts executing over here.
So here we have the model summary gradient zero parameters.
The parameters are in millions.
Okay.
So you can see the output.
Currently there are no detections.
But uh let's see as we go ahead in the frames uh we have any detection or not.
Okay.
So now you can see here we have the detections.
And you can see over here a unique ID is being assigned to each of the person.
So here we have the person.
Now you can see over here the 19 is put unique IDs for this person.
Two unique IDs for this person 28 unique IDs for this person which disappears.
Five unique IDs for this person 17 unique IDs for this person, 15 unique IDs for this person.
And here we have detected the handbag.
So the 30 unique ID is for this person.
The 14 unique ID is for this person, 12 unique ID is for this person 11 unique IDs for this person,
nine unique ID is for this person.
Now the eight unique IDs for this person, 16 unique IDs for this person, 21 unique IDs for this person.
So each of the person is being assigned a unique ID over here.
Okay.
So if you want to draw the trails and plus you can see we have detected a handbag and we have assigned
ID 30.
So as well.
So if you want to draw the trails uh you can just write over here.
Dash dash, draw, dash trails.
And if you want to select any specific class, like if you don't want to detect Hand-bags in the video,
you can just write classes zero.
So zero is for the person class.
So if you want to know more about the class so you can see zero is for the person, class one is for
the bicycle, class two is for the car.
Class three is for the motorbike class.
So here we have listed 80 different classes.
So it starts from zero and ends at 79 okay.
I won't hear in the classes.
You will assign the ID for which specific class you want to detect.
So if you want to detect all the classes, you can skip the classes over here.
But if you want to detect the specific class, you can just pass out the ID of that specific class or
multiple classes.
You can also create a list and pass the ID of the multiple classes that you want to detect.
So now it will not be returning Hand-bags or any other things.
It will only be detecting the person class because we have passed the ID classes as zero and we have
draw that passed draw trail.
So it will be drawing the trails over here as well.
So let's see now.
Currently there are no detections.
After a few frames initial frames we start to see detections usually okay.
So now you can see over here we have the detections.
Uh okay.
Okay.
So now you can see that red center center as well.
So these are the trails which will not be drawn.
So now you can see as the persons go ahead the trails increase as well.
So you can see over here it's a bit slow.
But now you can see that trails is also increasing.
So these are we are just tracking the path that the person is following.
You can also do it for other classes like car.
You can track the path the car is uh going on and so on.
So now you can see I am drawing the trails over here as well.
We have assign a unique ID using object tracking algorithm.
We are also tracking the path the person is following over here as well.
Okay, so this is all we have done in this code.
We are using object tracking algorithm.
We are tracking each of the detected object and we are drawing the trails over here.
So using trails we are tracking the path like you can see.
So this trails length will increase as the person goes ahead.
You can see over here.
These are the trails that we have drawn, and these trail length will increase as the person goes ahead
further as well.
Okay.
So now you can see over here.
So I have run this script on the Google Colab free GPU as well.
So.
So that I can use the GPU and show you the how fast it works.
So I have the demo video out over here with me as well.
So let me just play that demo video.
So I have run this script on GPU, so you can see the processing comes very fast.
And we have uh, it offers good, uh, frame rate at one GPU.
So you can see we have a good number of detections as well.
And it's the speed is very good as well.
So let me show you the output video over here.
So now you can see over here we are able to detect the person we are, have assigned a unique ID to
each of the person as well.
Plus I'm not drawing the trails.
You can draw the trails if you want as well so that you can track the path the person is following or
any other object.
So in this tutorial we have learned how we can track, uh, each of the detected object, uh, by assigning
a unique ID to each of the objective, uh, detected object.
So in this tutorial we have integrated yolo v9 with Deepsort object tracking algorithm.
And we have, uh.
Attract and detect each of the object as well, and we have assigned a unique ID to each of the object.
Plus we have also, uh, drawn trails to track the path the object is following.
So that's all from this tutorial.
Thank you for watching.