In this tutorial, we will learn how we can do object tracking using YOLO v9 and sort algorithm for
object detection.
I will be using the Yolo v9 algorithm and to track each of the detected object I will be using the sort
algorithm.
So let's get started with it.
Over here you can see that I have created a new project by the name Yolo v nine sort recording, and
I have just created this project in my local directory, uh, in my local directory so you can create
a project, uh, new project in any of your local directory as well, using PyCharm or any other IDE
as well.
So I have listed all the steps which I will be following in this tutorial in this notepad.
So over here in the step number one we will clone the Yolo v nine GitHub repo.
So I will just go over here in the browser.
So here you can see the Yolo v nine GitHub repo.
So to clone it I will just click over here and copy the URL to the clipboard.
Okay.
So now it's copied.
And now I will just go back to the PyCharm from here.
And I will just open the terminal terminal over here.
And to clone this complete repo GitHub repo into my this into this project yolo v nine sort recording
I will just write it clone.
And I will add the URL of the benign repo over here.
So now you can see that the cloning has started and it's completed.
So now I will if I just refresh this over here.
Okay okay.
So no no need to refresh.
We have this yolo v nine wrapper over here.
So now I will just set this yolo v9 wrapper which have cloned into my project as my current directory.
So I will just set this YOLO v nine folder as my current directory.
And you can see we have all the files over here as well.
So to set this as YOLO benign folder as my current directory I will write CD which stands for current
directory and I will just write YOLO v nine over here.
So this YOLO v nine folder is being set as my current directory.
So in the step number one uh we need to install all the required packages.
So over here in the requirements.txt file you can find all the required packages that are required to
run this.
Uh, to run that detection and tracking script successfully.
So which include OpenCV, Python, matplotlib, NumPy, SciPy, Torchvision, Tqdm and pandas, TensorFlow,
js and all the required packages are listed over here, so these packages are required to run the detection
and tracking script successfully.
Or if you want to train your object detection model on any custom data set, you need to install all
these packages so we can install all these packages one by one.
Or there is another way we can do which is quite easy way.
So you can just write over here pip install minus r.
Required.
Men's dot txt.
So when you write pip install r requirements.txt it is basically install all the packages that are required
listed in the requirements.txt file.
So it will install all these packages that are listed in this requirements.txt file.
So I will just click on enter from here.
And now all the packages that are listed inside the requirements.txt file will get installed.
So this might take a few more seconds for the process to get completed and all these packages to get
installed.
You can see that the packages installation is currently in progress.
Now you can see over here all the required packages are being installed successfully.
Next if I just open this up.
So now we need to download a sample video from the Google Drive link below, or download a sample image
from the Google Drive link below.
So I will just copy this link from here.
So to download any image or a video from the Google Drive link, we need to install the G down uh library
as well.
So first I will install the G down library.
So I will just write pip install g down.
So this will install this package.
So this help us to download uh any video image or any file from the or any zip file or any other file
from the Google drive directly into this, our local IDE, or either in Google Colab, notebook, Jupyter
Notebook, Visual Studio or PyCharm or any other IDE as well.
So g down stand for Google Drive download.
I like this thing, so I will just uh, plus I will add this up in the requirements.txt file as well.
So in any case, when you install the requirements dot txt file.
So this package also gets stored along with this as well.
So I g down here as well.
Okay.
So next I will just open this up, copy this link, and I will just add this link over here and click
on enter.
So this will download a sample image from the drive into this our project.
So over here image one dot jpg.
So if I let me see.
So here you can see we have the image one dot jpg get installed.
So.
Next, we need to download sample videos from the Google Drive link below.
So first I will just copy this Google Drive link and enter.
So this will download a video from the drive into this PyCharm IDE.
So test dot mp4 is the name of the demo video.
So over here you can see.
We have the test walked MP4 video over here next if I just go over here.
So when you install the second demo video over here.
So I will just add this link over here.
So this will take a few more seconds before we have it.
So now you can see we have the test three dot mp4.
This is the other demo video.
And if I just refresh this up here.
So you can see here we have test three dot mp4 test dot mp4.
So we have uh downloaded the sample image and sample videos from the Google Drive directly into this
PyCharm IDE as well.
So that's been done successfully.
So download the sample image, download the sample videos from Google Drive link below.
So now we need to do object detection on image.
So as we uh first uh to do object detection on image.
So you just need to go into this yolo v nine GitHub repo over here.
Uh you will check the releases version 0.1.
And over here you will see the model weights file over here.
So you can just click over here.
And the model weights file will be downloaded.
So if you just click over here.
So this will download the model weights.
So I will just cancel this out because I have already have downloaded the model weights.
And let me show you.
So over here you can see over here we are the model weights.
So here you can see we have the model weights.
So.
Copy this file name over here, and I will just go into my project over here, and I will just add the
weights file over here as well.
So you can see we have the model weights over here.
So that's been done.
So next we will just go to the code from here.
And over here.
So if you just go below down below.
So to do inference with YOLO v nine, like if you want to do object detection on image with yolo v nine,
uh, by using the pre-trained model weights, you will we will using detect test.py script file and
in the source will pass the image path.
And then we have the pre-trained model weights over here.
So that's all.
So I will just go over here.
I will just write Python.
You take that dashed.py.
So here we have the detect-dual.py file.
And then we have the source as image one dot jpg okay.
And then we have the weights as.
Yolo b9 that c dot b2.
So you can see here we have the your weights file over here.
Here we have the input image file over here.
And if I just show you the input image.
So this is our input image.
And we will be doing object detection on this input image.
Okay.
Plus I don't have the GPU in my local system, so I will be using CPU.
So device, CPU and um, anything else.
So I want to see the output image.
So I will just write dash dash view image.
Plus I will just put it-dual.py.
And I will just make one change in the code.
I will just change it to zero.
So the output will be image will be displayed for infinite time if I just write one.
So the output image will be displayed for one millisecond, which is very small and we will not be able
to see the results.
So now it will be displayed for infinite time.
Okay, so let's run this up now.
I hope it will work fine.
But if in any case, in theory, if there is any error, we will try to fix this up.
So now we are doing object detection on this input image.
Image one dot jpg.
And let's see whether we are able to detect different objects inside this image or not.
I'm using CPU so this will take some time.
Like with GPU, the inference will be very fast, but with GPU the but with CPU the inference is very
slow.
So over here you can see that we are able to detect burst persons or traffic lights.
Like you can see the persons are bit far away and very blurred.
But the Yolov5 model is able to detect successfully.
We are able to like bicycle backpack, hand bags.
So the results look quite impressive from here.
Okay.
And bag is also detected.
So.
Okay, so if you can press any key, the, uh, it will the script will close up and the output image
will also be closed.
So next we will see how we can do, uh, object detection on a video.
So.
Let's make it to one again.
So each frame will be displayed for one millisecond.
And over here I will just add the path for the video.
So over here I will just write.
Um.
Best free DAW MP.
So now we'll be doing object detection on this street MP4 video.
And let's see how does it goes.
So we have done object detection on video.
We have done now we are doing object detection on.
We have done object detection on image.
Now we are doing object detection on video.
And next we will be doing object detection object tracking using yolo v9 and sort algorithm.
So currently we are doing object detection on video.
Okay.
So.
Okay.
So over here you can see that we are able to detect the cars.
Uh, so these are the false positives.
This is the false positive.
And here you can see that we are able to detect the cars.
So here you can see we are also able to detect the cars as well.
So the detection is very slow because we are running it on CPU machine.
But if you use GPU the detections uh the frame rate will be very high.
Currently the frame rates are very low.
Uh, but if you test it on GPU, the frame rate will be around 5 to 6 fps frame per second.
But I'm testing on CPU so the frame rate is very low.
But you can see over here we are able to track the cars very successfully.
And these three.
And this truck is pos positive.
So false positive means like there is no car, but it's detecting the car.
So this is a false positive prediction.
Like there doesn't exist any car over here and there doesn't exist any truck, but it's returning as
a truck.
So this is a false positive okay.
So like in this way you can test on the video as well.
So if I just press control C so this will interrupt this process.
So now we have done object detection on image.
We have done object detection on video.
Let's move on and see how we can do object tracking using the Sort algorithm.
So if you just go over here and if you just uh right on Google.
So if let me just show you.
So on GitHub.
So you just need to open the.
This first link.
So sort stand for simple online real time tracking of multiple objects in a video sequence.
Okay, so you just need this sort.py file.
So here we have the saw.py file.
So if you just can download this this file will be downloaded.
But I have made some changes into the saw.py file.
So if you use this by default saw.py board you will be in the output of.
If you use this saw.py board in the output, you will be getting the bounding box coordinates and the
unique IDs for each of the object.
But I have upgraded this saw.py file code.
And now in the output you will be getting the uh bounding box coordinates unique IDs.
Plus you will also be getting the class names.
So if you use this uh saw.py code over here, you will not be able to get the class names, but I have
updated this code.
Now you will be getting the, uh, classes name unique IDs, because in object tracking we assign a
unique ID to each of the object, and plus you will be also be getting the bounding box coordinates.
Okay.
So it's your choice.
Either you use this, but I have updated this code a bit.
So I will be using that code.
So let me add the code over here.
So over here.
Okay.
So who are you?
Here?
You can see we have the updated source code and we will be using this.
Um, we will be using this function sort over this class over here.
And we will be using the update function over here.
Okay.
And from here we will be getting the bounding box coordinates, unique IDs for each of the object.
And along with this we will also be getting the class IDs or the class names.
Okay so let's go ahead with this.
I am creating a new file detect-dual-tracking.py file.
And let me add that file over here.
So I have just created a new file detect-dual-tracking.py.
And over here I will be writing the code to do object tracking using sort algorithm.
The object.
The code for object detection using yolo v9 is already added.
I just need to add or integrate the object tracking code into this.py file, and we will be using the
doing the object tracking using sort algorithm.
Uh, in object tracking, what we do in object tracking, we basically assign a unique ID to each of
the objects.
So here you can see we have the sort.py file.
And from the sort.py file I will be importing all the functions.
And to import all the functions we will just write star.
So it will import all the functions from the sort.py file.
Next I will be just writing tracker.
So if you just open this up for.py file over here.
So I will be using this class uh which is this.
So I will be using this class sort over here.
So I will just write sort.
And if I just see.
So here we have the maximum age.
So in maximum age or what we define.
So for example if an object is lost or is there if there is any occlusion if the object is facing.
So if for example, in simple words, if an object is lost, how many frames our sort algorithm will
wait before assigning it a new ID.
So for example, if an object is lost in the frame of a video, how many frames the sort algorithm will
wait before assigning a new unique ID.
So if I just write this value 20 over here.
So our saw if for example, if I just write 20 over here.
So if the car is lost in the video frame, our sort algorithm will wait 20 for 20 frames before assigning
it a new ID.
So maximum age we define if an object is lost in a video frame.
How many, uh, frames our sort algorithm will wait before assigning it a new unique ID.
So if I just write 20 over here so our sort algorithm will wait 20 frames before assigning, uh, the
object object a new UK unique ID.
Okay.
So.
So our sort algorithm will be waiting 2420 frames before assigning, uh, that object a new unique ID
if the that object ID is lost.
Then we will define minimum hits as three and IOU threshold.
I have already defined this.
What is IOU threshold at the start.
So we'll just keep this value 0.3.
Okay, so next what we will do is, uh, I will just go down below and, uh, where I will start writing
the code.
So I will just write the code.
How we can do object tracking using sort algorithm over here.
So let's get started with it.
So first of all I will just create an empty numpy array of size.
Zero six.
Okay, so from here from the Yolo v9 algorithm, we will be getting the bounding box coordinates confidence
score and the class name.
So here from here is the output of the Yolo v9 algorithm.
So if I just show you over here.
So from the Yolo v9 algorithm we get the uh bounding box coordinates class name and confidence score.
So we get the uh top left corner bounding box coordinates which are x one, y one.
And for each of the detected object we get the top left corner bounding box coordinates x one, y one,
and the bottom right corner bounding box coordinates which are x two, y two.
So and plus we also get the class name and confidence score.
So for each of the detected object we will be getting the top left corner bounding box coordinates X1Y1
and bottom right corner uh bounding box coordinates X2Y2.
So for each of the detected object.
So over here I will just write for x, y x, y.
So these are this is the X1Y2X1Y1.
And this is X2Y2.
Okay.
So from your lovin algorithm, we get the.
Boundingbox.
Coordinate confidence score and class name.
And we weren't getting this from infections.
Okay.
So let's separate the x1, y1 x2 y2 values.
So from here the output is basically in the form of tensors.
We need to convert the output from tensors into integers.
So now we are just converting the output from tensors into integers.
Then we just need to calculate the confidence score.
So we just need to have math dot ceil.
And we need to convert the confidence value into integers as well.
You can call this as confidence score over here as well.
Last will also be getting the class name over here.
In.
So.
So these are the, uh, output which gets which we get from the Yolo v9 algorithm.
Yolo v9 algorithm gives us the bounding box coordinates, confidence score and the class name.
And now we'll create an array which we call grant array.
And in this area we will have the.
Bounding box coordinates.
Values.
And we'll also have the confidence score over here.
And the class name.
Over here.
Okay, so let me show you how this looks like if I just print this out over here.
Uh, I think so.
We don't need to view the output image currently.
And.
Right in Thai restaurant on this other video and let me show you how this looks up.
So in any case, if there are any error appears, we'll try to sort this up as well.
Okay, so to run the sort algorithm skills step, we need to have the filter pi package installed as
well.
So I will just uh pi pip install filter pi.
So I will just right add this up in this as well.
Filter pi.
So we just need to install this package individually as well.
So let's run this up now.
Hopefully it will run now.
Though it has started.
We will not.
So now you can see over here okay.
So okay.
So here you can see that, uh, so for example.
In the first frame we have 28 persons detected and there is one handbag.
So in the first frame we have detected 28 persons and there is one handbag.
Okay.
So now you can see this is 26 ID which is for handbag.
So these are the x1 y1 x2, y2 bounding box coordinates.
This is the confidence score.
And this is the class name.
Okay.
So in the same way we have already written over here like we have the bounding box coordinates and confidence
score and class name.
So like you can see we have detected 28 persons and one handbag.
So for all the 28 persons we have this coordinate.
So for each of the detected object like for the first detected object we have this bounding box coordinates.
And we have this confidence score.
And this is the class name for the second object uh detected object we have this bounding box coordinates.
And this is the confidence score.
And this is the class name for the third detected object.
These are the bounding box coordinates.
This is the confidence score.
And this is the class name.
So we are just getting the bounding box coordinates confidence score and class name for each of the
object one by one over here.
So let's go ahead with this up as well now.
So now I will just write detection is equal to NP dot p stack.
So we will be doing vertical stack.
Deductions.
Country.
So now let me show you what this vertical stack does.
So if I just write.
Then.
Actions.
Okay.
So let's run this script again.
So let me show you.
We have the for the first frame.
Okay.
So let me just roll it up now.
So now you can see that these are the bounding box coordinates for the first detection.
Okay.
This is the confidence score.
And this is the class name.
Okay.
So now like you can see that this is the first uh array.
Okay.
And then for the so now you can see that these are the bounding box coordinates for the second detection.
And this is the confidence score.
And this is the class name.
But you can see over here we have also added the first detected object bounding box coordinates.
Contnued confidence score and class name into this numpy array as well.
Now if I just code over here.
Now these are the bounding box coordinates confidence score and class name for the first two detections.
Now here you can see we have for the third detection.
And I have added the bounding box coordinates confidence score and class name for the first two detections
as well.
So we are doing vertical stack over here.
And here you can see these are the bounding box coordinates confidence score and the class name for
the for detection.
But I have added the bounding box coordinate confidence score and class name for the first three detections
as well okay.
So this is how it works.
Now I will just write.
Tracker.
So you can see that I've initialized the sort, uh, algorithm over above over here.
So we initialized over here.
So we have tracker.
Okay, so now I will just write over here tracker.
So I will be using the update function inside this sort.
Dot py file.
So I'm using the class sort and I will be using this update function over here which you can see.
So.
Okay, so let's use the tracker dot update.
And what this function accepts.
So this function accepts reductions.
And over here I will passing this reductions over here.
Okay.
So so now before we go ahead, uh, the, the sort algorithm.
So the input of the Sort algorithm, we have passed out the bounding box coordinates confidence score
and class name.
So what output we get from the sort algorithm okay.
So if I just open this from here.
So from the sort object tracking algorithm we get out we get the bounding box coordinates identities,
unique IDs for each of the object and class name.
So we don't get the confidence score from the sort object tracking algorithm output.
We get the unique IDs for each of the detected object.
Okay.
So I will just write.
So if there are any detections.
So this will return us the bounding box coordinates.
So if you just print this tracker detections.
Uh, let me just print this out first and then we will discuss.
Okay?
Yeah, sure.
Soon.
Well, let's run this up first.
So this will take a few seconds before it starts executing.
Okay, so here we have it.
So if you just see over here so the first four have the bounding box coordinates.
Okay.
So if I just write over here.
So.
In the first four, we have the bounding box coordinates.
And these are 0012345678.
The eight item contain the unique IDs.
Okay.
And.
01234, and this fourth item contains the class names.
Okay.
Or these are the class IDs you can say.
So we can simply write get grease.
So this four item, uh, 01234 contain the class IDs.
So now I have just created a function over here by the name draw those boxes inside the draw boxes.
Function.
Draw boxes function.
We pass down the bounding box coordinates and entities and the class ids.
For example, the class ID for the person class is zero.
For the bicycle it's one.
Okay.
And this, uh, what?
In the draw box function I've just written using OpenCV.
Uh cv2 dot rectangle function OpenCV cv2 dot rectangle function.
We are drawing the bounding box coordinates or rectangles around each of the detected object.
Plus we are also creating a rectangle above the detected object, where we will be adding the IDs and
the class names over there as well.
So in the label we will have the id and the class name, and we will be drawing a circle at the center
of each of the detected object as well.
And let me discuss this in detail okay.
So I will just calling the draw boxes function over here.
Well, this is the input video frame.
Morning.
Most gardeners.
Unique IDs for each of the detected object and the categories.
Okay, so.
So now you can see I have this path to the draw boxes function.
And here we have this function.
So here we are just finding the center point of the bounding box.
We are just uh have finding the unique I and uh, giving the unique IDs to each of the object which
we have or which we are already getting the sort algorithm, we are drawing drawing rectangle around
each of the detected object.
Plus we have just created this function class names.
So if we get the object id zero this mean it represent the detected object is the person.
If the get uh class ID or categories, which we have called this as one, it is a bicycle restricted.
If the class ID which we got is two.
So the car is detected.
If the class ID is three then the motorbike is detected.
If the car class ID is four then the aeroplane is detected.
If the class ID is five, then the bus is detected.
If the class ID is 80, then the toothbrush is detected.
If the class ID is, uh 79, then here I draw restricted.
Okay.
So.
You can see over here.
Um, so here we have the class IDs.
So, so in the categories we have the class IDs.
And this ID represent a unique ID which we assign to each of the object.
Okay.
So in object tracking we basically assign a unique ID to each of the object.
So you can see that in this cat we cat cat we have the class ID.
And when this we pass the class ID over here.
So if the class ID is zero then we will be calling this function over here.
And it will tell us that this is a person class and so on.
Okay.
And in the label we have the unique ID that we have assigned to each of the object plus the class name
over here.
And then we are just drawing a circle around each of us at the center of each of the detected object.
Okay, so now I will just run this script over here.
So plus I will just write view dash image and let's run this up over here.
Let's see if we are able to detect the objects and assign a unique ID to each of the detected object
using the Sort algorithm or not.
So over here.
That's great.
So now you can see over here, uh, each detected object is being assigned a unique ID six 232 2314
1710 2925 2119 eight 1813 1516, 1231 2020.
So you can see that using object tracking algorithm, we are able to assign a unique ID to each of the
detected object.
And that's amazing.
That's great.
Okay so I'm testing on CPU.
So the processing is a bit slow.
Let me just test it on other video as well.
So now you can see over here we have assigned a unique ID to the car as well.
So this is the wrong detection but a unique ID assigned to each of the object.
Because we assign a unique ID to each of the detections using sort algorithm.
Okay.
So one for two.
So that will look pretty fine as well.
Like you can see let's see if we are able to detect this car or not.
So you can see the car is detected and unique ID seven is assigned over here as well.
And this works pretty good.
So if you want to test it on the live webcam feed, you can, uh, simply right over here in the source
zero.
So if you write source zero, you will be able to test on the live webcam feed.
And if you want to test on the external webcam, you can write one.
And if you want to test, uh, test on the uh, like IP camera.
So you can just pass the IP, um, link over here.
So if you want to test on the, uh, your laptop webcam, you can just write zero.
If you want to test on external webcam, you can write one over here.
So if you want to test, uh, if you want to detect only specific classes, like if you just write over
here like test dot mp4.
So if you want to like you want to only detect uh person class okay.
So the person class ID is.
Zero.
Okay, so you can see over here, uh, if I just show you the person class ID zero.
So I will just write.
Class is zero.
So this will only detect the person class and no other object will be detected.
So you can do this as well.
Oh, this will take few seconds before this script gets executed.
Uh, one thing we can do is.
You can simply remove the sperm.
So let's see how our output looks like.
Okay, so now you can see over here, uh, we are only detecting the person class.
No hand-bags.
No other thing is, uh, detected over here.
Okay, so now you can see that we are only detecting the person class.
No hand-bags or no other things are being detected.
So you can select any specific class which you want to detect in a video frame feed as well, or in
the live webcam feed.
So that's all from this tutorial.
Thank you for watching!
In this tutorial, we have learned how we can integrate object detection with object tracking.
And here we are using, uh, sort algorithm to do object tracking.
And for the object detection we are using the Yolo v nine algorithm.
Thank you for watching.
Bye bye.