Hello everyone.
In this video tutorial we will see how we can integrate Deepsort object tracking with YOLO.
So this is the complete end to end tutorial.
I will try to explain the complete code of the prediction.py file.
So first of all.
It can start and open.
PyCharm You can use any other for this project.
You can work on Visual Studio, Spyder or using Anaconda Prompt.
You can also implement this project.
So it all depends on your choice.
So here I'm creating a workspace for this project, like you can see here, repairing workspace.
So as you upgrade workspace prepares, I will go to file.
It can create a new pressure.
Thank.
And here I will choose the file object tracking YOLO V8, which is the name of the folder.
And here I will also choose object tracking using YOLO.
So that's fine.
They can create this window.
Okay.
So we have created a new project now.
Well, next I will go to my GitHub repo.
I will share this link with you as well.
So you just need to as this GitHub repo contains the code for your deep sword object tracking.
So we will be using this code to implement this project.
Object tracking.
I will try to explain you the complete code as well.
So first of all, let's run the code while we are doing predictions for a demo video.
In that time I will explain you the complete code as well.
So first of all, let's see the steps to run the code.
So first we need to clone the repository.
Just click on copy over here and let's clone this GitHub repo.
Well.
Yes, exactly.
We'll just paste this over here and just click on Enter.
So now you can see that that rapport is being blown.
So it might take two more seconds.
So let's wait until the rapport gets blown.
Okay, search 67% and now soon it will be hundred percent as well.
75%.
82%.
87%.
100%.
Okay, so now we have clone the GitHub repo over here.
You can see that YOLO V8 Deepsort object tracking.
Now let's look the next step we need to do.
So in the next step, we need to go to the clone repo folder.
So let me show you what is this?
So currently you can see that this project Object tracking YOLO V8 is set to our current directory.
We need to set the current directory as this clone folder name, which is YOLO v8 Deepsort object tracking
should be the name of our current directory.
We need to set this as current directory over.
So just copy and this and paste what we have.
From copied from the GitHub repo.
So now you can see that this folder is being set as our current directory.
Okay.
So in the next step we need to install all the requirements or the dependencies so that we should not
face any error when we run the prediction that this library is not installed or this library is not
installed.
So by running this cell, by running this one inch, install all the required libraries and dependencies.
So just copy this from here and just paste it over here.
So now all the required libraries or the dependencies will be installed as I have run this port previously
as well.
So you can see that requirements are already satisfied.
But if you are running this port for the first time, then you need to install all the libraries.
So the dependencies are now installed.
So guys, as we are doing detection.
So we need to go to that predictions folder inside this repo which is Ultralytics, YOLO.
We hit the deck.
So this is the detection folder which contains the training step validation script and prediction script.
So as in this tutorial I am doing only a predicting uh, on the pre-trained model, so I will be only
using the prediction script.
I will not be using the training or validation step.
So let's set this folder, detect as a word current directory.
So just copy this from here and control V over here.
So this has been set our current directory.
Now we can run the prediction training or validation script where we will only be focusing on the prediction
script.
Next, we need to download the Deepsort files from the Google Drive.
Or just copy this from here and just paste this over here and click on Enter.
So you just need to go to right click over here and.
Okay, taking time.
We just click on download.
So now it first it will zip and then it will download.
It will not take much time.
It will take around 30 seconds to 40 seconds more.
So let's see what it will not be very time taking process.
This the file is zipping and very soon the file has zipped.
Now the download has started.
Okay.
Song.
I could just go to the showing folder.
Just copy this from here and just.
This is our folder which we have created for this project.
I will paste it over here.
I will unzip this first extract files.
Okay.
So now I will get go inside this folder.
Okay, So just copy this from here.
And just go to your deep sort of tracking ultralytics Yolo v it detect and just paste this folder over
here.
This is the Deepsort folder as we are implementing object tracking using Deepsort.
So we need to have that deepsort files.
On our side.
So we have just downloaded the Deepsort files which are necessary to implement object tracking using
Deepsort.
So if we are using a providing object tracking using SORT algorithm, we will require to have the sort
algorithm files.
So here as we are implementing object tracking using deepsort.
So we need to have that deepsort files over here so you can see that Deepsort PyTorch files are over
here.
Okay.
So.
But I have completed this step which is download the Deepsort files from the Google Drive because I'm
implementing object tracking using deepsort.
So I need to add that deepsort files.
So after downloading the deepsort zip file from the drive, unzip it so I've unzipped it and place into
the YOLO detect folder.
So I have completed this step.
So now we need to implement tracking using on some demo video.
So we need to have the demo video as well.
So let's download the demo video by just popping this command.
So I have placed the demo video on my Google Drive, so by just copying and pasting, this command will
download the demo video from the Google Drive into the detect folder.
Just paste this one over here and click on and drag.
So now the demo video will be downloaded like you can see that it will be appearing here.
Let me refresh it so you can see the test.
Dot MP4 is the name of our demo video.
Now let's run the script for YOLO V8 Object detection plus tracking again.
So just copy this over here so you can see that we are using YOLO V8 model over here.
So we have 5 to 6 different modes or sorry, five different models in YOLO, V8, YOLO V8 and is the
smallest, but it is.
It is fast and very less accurate.
Yolo V8X is the largest, so yolo be eight x is more accurate, but it is very less.
Its speed is very less as compared to the other YOLO models.
So we are using a medium because V8 model which is around the other V8 and or you can use your the V8
MPV as well.
Just copy this from here.
And.
Is the script over here and click on Enter.
So here we have passed the name of our demo video file.
Here we have the model name and show is equal to true means.
We want to check the output in the runtime.
Like if show is not is equal to true.
We don't write this, then we are the demo after processing the complete video or after running detection
or tracking on the complete video, it will be saved into my required folder, but I want to see the
output at the runtime so it and show is equal to true.
So this script is now starting to execute, so it might take some time.
So in the meanwhile I will try to explain with the concrete support of prediction.
So now you can see that the script has started to run.
So like you can see that we have a bounding box around the object.
We have a unique ID with each of the object, like you can see two car, five truck.
So this is detecting it wrongly.
But if you use a larger model of your V8, so it will give more accurate results.
So plus you can will be seeing the trails as well.
You can see these trails like you can see this brown line over here.
So you can see that trails and unique ID as well.
So we have implemented object tracking, but let's see the code and see how does this all process is
done.
So let's see the code so you can see that we have the unique ID and the tracking trails with each of
the detected object as well.
So let me show you the import of the code.
In the meanwhile, this script will be run completely.
Then we can look the output as well.
So.
Guys in front of you, you can see the Predict Pi script.
We have executed our demo video on this predict dot pi script.
So let me explain you this whole code in a best way I can.
So first of all, we have imported the Hydra library.
Uh, in the next step, we have got an import torch.
So as it is built using pi dot.
So to perform object detection tracking, using YOLO, we need to import the module so you to use the
PI torch library.
We do import torch.
So to check the version of the PI torch library, we can do print torch dot dash dash version so we
can check the version of the torch library.
So to.
If you face an error in import torch like this module is not available.
So you just need to go to the Chrome store and browser which you have just right pi torch over here
and just go to the pi.site, click on Get Started.
So if this module is not available, you can just go over here pi Torch build.
So you will select the stable, build your operating system which you have, you can select, plus you
can select PIP a language python, and if you have a GPU, you can select you deliver point six 11.7.
But as I don't have a GPU or CPU so I will select on CPU.
So you just need to copy this from here.
Just click on copy and you can just paste this command in your terminal and click on Enter.
So this will install the PI torch library or and when you do import torch, you will not again face
this error.
Next we have import torch dot backends, dot cuda and then so basically torch dot backends control the
behavior of various backends that PyTorch support.
So these backends include torch dot backends, dot dot dot backends, dot u n.
So there are multiple backends.
So bus just have mentioned two over here.
Next we are importing from numpy import random.
So numpy basically random is a module present in the numpy library.
So this module contains the functions which are used for generating random numbers.
So we are using this module to basically randomly generate color palettes for our different detected
objects.
We are not assigning a color palette to each of the objects.
We have assigned a specific color palette to some specific object.
But as we are running this script, our Pre-trained model is trained on coco dataset, so it contains
around 80 classes, so it will randomly select a color palette for each of the object and sign it.
So to randomly select a color better, we have imported the random library.
So here we have importing some required libraries.
Which are these?
Next, as we are implementing object detection and tracking, we need OpenCV Python library, which
is import cv2.
Here we are importing the deepsort PyTorch libraries required.
So.
So these libraries can be.
For example, if you order Deepsort PyTorch folder and you did go to utils and inside the.
parser.py file.
You will find the configuration files and if you go to deepsort or this folder, you will find the Deepsort
folder over here which is deepsort.
So we are importing all these required libraries of the deepsort into our main script file.
predict.py file.
Next we are initializing or we are importing from collections.
So tkeo is a double ended queuing python is implementing using the module collection.
So why?
So you can have a question that why we need the queue.
Why we can't use list.
So for our list in the cases we are need a quicker abandoned operations from the both end of the container.
So here we are using when basically we add an entities into our list.
So when the object disappears from the frame, we need to remove the object ID from that list.
So and and a new object appears in a frame.
We need to add that object ID into the list so for using the queue we can have a quick add and pop operation.
Pop is basically a removal operation, so when we assign a detect an object, we assign it a unique
ID When the object disappears from the frame, we need to remove the ID, and when the new object appears
in the frame, we need to get the we need to assign that ID so to assigning the ID or removing the ID
when the object disappears.
So this is a quick and pop operation which can be done using the queue.
So next, if we want to convert our list into an array, we use a numpy library.
So now we are defining the color palette because we will have a wide variety of tracks because they
are different objects.
So we have establish a color palette over here, so the color of the bounding boxes and the tracking
trails will be the same for each of the objects.
So like you can say if we have cars, so the cars bounding boxes and so on, the cars bounding boxes,
color and train tracking previous color will be the same.
So let me show you this by an image so that I can better explain this point.
What?
I've just taken this image from the output video which is displaying over here.
So if you can see over here, let me pick up and say, no, it's one or something like, no, this is
too much.
That is fine, but there should be some good one.
Okay.
More Saturday night, girl.
Okay, so that's good.
So you can see that on the cars.
You can see over here have a like a light pink color you can say.
And the bounding box color is also pink.
And these trails, like you can see that these trails color is also pink and the bounding box and the
labor label color is also being over here as if you can see that there is a truck.
So you can see that truck bounding box color is brown and this label color is also brown.
So if we have some other object here as well, so its color will also be disciplined and dreams will
be of that same color, which is of the bounding box color.
So for each object like this is that truck and this is a car.
So for each object we have the same bounding box color and the trails color.
Okay.
While if the for example, there is a truck, the truck will have a brown color and its trails will
have the brown color.
If there is a car, the car will have the pink color and its trail will have the also have the pink
color.
Okay.
So here we are defining the color palette because we will have a wide variety of tracks, like we can
have around 80 different objects in a video.
So we want to establish a color pattern.
So the color of the bounding boxes and the tracking frames will be same for each object which we have
seen.
So here we are, initializing a dictionary by the name data, okay?
And here we have just initialized the sword as none.
So here we are, initializing the deepsort and configuration files inside this initialize tracker function.
So inside this track, inside this initialize tracker function, we are initializing the deepsort and
all the required configuration files.
Okay, so here we have a first function define x, y, x, y to x, y w.
So.
So basically.
Because jewelry here gives us the predictions in the form of corner coordinates.
But Deepsort requires the prediction in the form of center coordinates and the width and the height
of the bounding boxes.
Let me explain you this why we need this function.
So, for example, let me just go over here and create a So, for example, this is a bounding box and
here we have an object inside it.
So the object can be anything.
So.
So you can say that this is our object inside the bounding box.
So.
This is our bounding box and this is our object inside the bounding box.
So.
So it will give us prediction that we right over here.
In the form of x1 y1.
Okay, so x1 y1 is basically the.
Top of this is the top corner, 2X1Y1 represent this top, top left corner.
Okay.
So you can in the output we will get this coordinate X1Y1 over here.
What is this?
And this is the bottom right corner.
This value will given by the desert, which is X2Y2.
Okay.
So only these two values will be provided by the model, like the the top left corner and the bottom
right coordinate corner coordinates.
So only these two coordinates values will be provided to us like this.
This coordinate value will be X1Y1 in the form of and this corner value, bottom right corner value
will be in the form of X2Y2.
So these two values will be provided by the YOLO V8 model.
But what Deepsort requires is Deepsort requires the prediction in the form of center coordinates.
Deepsort wants us the.
Let me show you that.
So Deepsort wants us this point these coordinates so X sleeps or wants predictions in the form of x,
y, z mean the center coordinates Deepsort only want the center coordinates and weight.
Weight and height of the bounding box is like, this is the height.
This is the height of the bounding box and the width of the bounding box.
Okay.
So it gives us.
Let me write in a more brighter color.
Okay, so.
Yellowbeard gives us prediction in the form of X1Y1.
This is the top left coordinate over here and the bottom right coordinate so it gives a prediction.
The form of x one, y one and X2Y2.
What Deepsort wants a prediction in the form of center coordinates like x, y, z.
And the width and the height of the bounding boxes like width and height of the bounding boxes.
This is the height.
And this is the width of the bounding boxes.
Okay.
I know I'm writing very poorly.
So in the next step, so after so we have created this function.
This will convert the x1, y1 and x2 y2 coordinates like these to these coordinates.
These coordinates and these coordinates.
So.
So this function will convert these two coordinates into the center, coordinates and width and height
of the bounding boxes.
So as you can see here in the define x one, y, x, y, z, two x, y function, we are converting
the bounding box output received from a V8 to a format that is compatible with deepsort.
Using this function, we convert the x1 x and y coordinates to center coordinates, which is x and y
and return the height and width of the bounding boxes.
So I hope you have realized this importance of this function.
Now we have a function defined x, y, z, 2X2.
And so.
So this function converts the x y coordinates into another format.
In this case, we are only looking for the Doppler width and height.
So this function basically gives us that top left coordinates.
Like let me show you.
So with this second version will only give this coordinate.
The top left coordinate, which is this one.
So only this coordinate will be provided.
Let me show you another color.
So.
Only this coordinate this one.
This point and.
And the width and height of the bounding boxes.
Okay, so this will give us a top left coordinate and width and height of the bounding boxes.
This function will provide but liquid as we are not using this function in this code so you can just
skip this.
So next we are computing colors for the labels.
So here we have defined some colors for the specific labels.
For example, there is a label of person which is zero label.
If we have object that is zero, it will means it represents a class of person.
So if we have a label zero, then we will assign this color palette to the bounding boxes and that fills
or as well.
So these are the trails.
When I'm talking about trails, I'm talking about these lines.
Okay.
These lines.
You can see over here.
These lines.
Okay.
So if we have a person object of class, so we'll assign this color to bounding boxes and the trails.
If we have a car, we will assign this color.
So this is light pink color which is being currently being assigned.
This is light in color.
And if the your motorbike is color will decide if you have a bus, this color will be a.
So if the detected object does not belongs to any of these labels or class, then it will randomly select
the color palette and it will show if the detected object is not from any of this class, then it will
randomly select a color and it will assign a in.
The output will get the attribute of that color.
Okay, so.
Next we have our draw our border function in the draw border function.
In draw dash border function.
We have written the script to create a rounded rectangle for the label.
Let me show you.
So.
So you can see over here, this is a rounded rectangle in which contains the 86 cent card.
So to create this rounded rectangle is rounded rectangle only.
I have written the script for the Draw Dash border.
So this draw dash border contains a square with the help of which we can create this rounded rectangle.
Okay, so next we have the dashboard function.
So in the box function we have the cv2 dot rectangle.
So Cv2 dot rectangles create a bounding box like this bounding box around the detected objects like
you can see here as well.
So you can see here as well.
So.
Uh, see, we're using cv2 dot rectangle.
We can create a bounding box around the detected object.
Okay.
And C1 C2 contain the center coordinates.
Okay.
So in the next step, we are calling the draw dash border function over here.
So this is the draw dash border function which I have explained previously.
So now we are calling draw dash border function over here to create a rounded rectangle like this rounded
rectangle.
And this one I've already explained.
And using cv2 dot put text using this Cv2 dot put text.
We are adding the text text in this drawer dash rectangle which is 86 and car.
We'll draw this border.
We create a rounded rectangle like this one.
And using cv2 dot put text.
We add this text like identity ID number or unique ID number of each object and the object name.
Okay, so this is on.
Now we have a draw boxes function in draw box six function.
I am calling the dash box function.
Let me show you its called.
So.
And let me see where it is called.
For him.
The draw dice boxes function we have called the box function.
So the box function will add the rectangle around the y function.
Box function will create this rectangle around the detected object like this rectangle square rectangle
and also add the this rounded rectangle as well and put the text into this as well.
So box function do thing three things.
First, it create a rounded rectangle.
It creates a rectangle around the detected object, which is a square rectangle.
Second, it adds a it creates a rounded rectangle using draw dash border function because we are calling
draw dash border function inside the dash box function plus it assigns a unique ID and the label name.
Okay, so.
Let's go up.
So to draw the bounding boxes around the detected object, assigning unique IDs to each detected object,
assigning labels will be done using the.
The dash box function which we have created over here.
And we are just calling this dash function function in this drawer, dash boxes function and this drawer
dash drawing.
Drawing will also draw the trails for tracking.
So drawing the trails for tracking script will be written in this over here, like this is the trails.
These are the trails.
These are the trails I'm talking about.
Okay.
So as we are doing detection frame by frame, so here I am checking the height and width of the current
frame.
Okay.
So here I'm just checking the height and width of the current frame.
Okay.
So we will store the unique IDs of the detected object until the object is in the frame.
Okay, so let me explain this.
You can see that each object is being assigned a unique ID 87, 53, 67, 68.
So all these IDs, unique IDs.
Basically all these unique IDs will be stored in a list.
Okay, So these will be stored in that list which we have created.
Like we have told you in the start, that using we can add and remove an item from the list.
Okay, so these will be stored in the list.
So until this specific object is in the frame, the ID will be assigned to that object.
When the object is disappears, for example, the yellow card is in the current frame, but after few
frames, the yellow card disappears from the frame.
Okay, so we will remove As the object disappears from the frame, we will remove the unique ID of that
object from the list and will also end.
For example, the yellow card disappears and the black card enters into that frame in our frame.
So we will assign the unique ID to the new object and we will also save the new unique ID of the new
object appearing in the frame.
Okay, so here we have written a script.
So if the object disappears from the frame, we will remove that object.
Unique id.
Okay.
Using dot pop key, we remove the ID of the object from the list if the object is no more in the frame.
Okay.
So next we will.
Well, now we will loop through the bounding boxes one by one.
So here we have the on the paw coordinates for each of the bounding box.
So x one and y one represents the top left corner and X2 and Y2 represent the bottom right corner of
the bounding box.
Let me show you.
I have ordered his plane.
But let me explain this as well.
So if we just clean this up it so that I can in a better position to explain this.
Okay.
So.
Okay.
So let me show you.
So we have you can see that multiple bounding boxes over here like one, two, three, four, five,
six, seven, eight, nine.
Okay, so one thing is.
This is basically x one.
Y one.
It is the top left corner and this is the bottom right corner and this represents x2 y2.
Okay.
So for each of the bounding box over here.
Like this one.
This one.
Okay.
Like for each of the bounding box over here, we have the four coordinates for each of the bounding
box.
Which is x1, y1, x2 and Y2.
Okay, so next we need to find the center of the bottom edge of the bounding box.
So let me show you.
So in the next step, basically, this is the center of the bottom edge of the bounding box.
Like this is the center of the bottom edge of the bounding box.
We need to find the center because we will draw trails from the center.
You can see that these are the trails.
So we need to find the center.
So here we have written the code to find the center of the bottom edge of the bounding box.
Next.
We will.
We need to have a unique ID for each of the project.
So using this script, we'll get a unique ID for each of the detected object.
Okay, so next, we can see if the new object appears in the frame.
So here we have assigned a new ID if the a new objects appear in the frame.
Okay, so now we will create a new buffer or a new list which has the maximum length 64.
So if so, let me show you.
So we have we have created a list which maximum length is 64.
So what we will do here is basically you can see this.
We have drawn the trails over here.
So we have saved this center points basically after getting the center point of this, like after calculating
the bottom center.
Let me try to explain this better way.
So after finding the center of the bottom edge of the bounding box.
So after I find the center of the bottom edge of the bounding box, I will save all the center points
of each of the frame.
Like there are 100 frames in a video.
I will save this center of the bottom edge of the bounding box of each of the each of sorry, of the
bottom edge of the bounding box of each of the frame in a list.
So basically the center of the bottom edge of the bounding box of all the hundred frames in which the
object is appearing is saved in a list.
Okay, which maximum length is 64.
So if a 65 fifth of the object appears, element appears it will remove the first and it will add the
65 into it.
So.
As the new elements appear that all the previous elements will be removed.
Okay, so if we set this line to 128 so one thing will happen.
This trains will go further long because after we save the center coordinates like the bottom edge,
bottom of the the bounding box coordinate and we draw the trails using these coordinates, which we
have saved.
Okay, so.
After saving the center of the of the module, the bounding box coordinates, we draw trees, which
maximum length is 64.
So if we change the maximum length to 128, this change length will go further wrong.
Okay, so now we are setting a unique color for each bounding box and rounding rectangles which could
be in the labor.
Okay, so next we have the object id object id contains the ID for the each of the object, like for
the person, the object ID will be zero.
So when we write names, zero.
So in the names we have the names for on the object.
So if we object, ID is zero over here.
So if there is a zero over here.
So it means that the object name is person.
If there is a one, then it means there is a car or a bus.
Okay.
So now here we are just putting the labels in a required format.
So you can see that we are appending the center points of the bottom edge of the bounding box like we
are appending these center coordinates into our data dequeue list so that a data list is maximum.
That is 64.
So.
Using this center coordinates would be saved in the data list.
And using these coordinates, we will draw the trails.
Okay, so now we are just using the foot box function to create a rounded rectangle.
To create this rounded rectangle and to add the text and to draw this rounded rectangle as well.
To create this rectangle.
Sorry, this is not the rounded.
We are using this box function to create this rectangle and this rounded rectangle and to put that text
over here as well, which is ID and label.
Okay, so now here we are, just drawing the trails.
You can you can see that we are drawing the trails using Cv2 dot nine.
So it basically loops through the data data center points and draw a line between the current and last
point.
So if, if the lattice maximum length is 64.
Okay.
So we have saved a 64 and the center points in the list, if the net is 64, we saved us 64 center points
in the list is 64 and we have the length of 64 trails of 64 center point if we change the maximum length
to 128.
Okay.
So this trails that will go even further.
Okay.
So in this way, all this process is implemented here, we will draw the line of the trails in next.
This is all the code for the.
If we go further.
So what we are doing here, we are just looping over all the detections because it gives us the corner
coordinates like x1, Y1, x2 and Y2, but Deepsort requires the center coordinates and the height and
the width.
So here we are, just converting the corner coordinates like this.
Let me show you.
Here we are converting the coordinate coordinates X1Y1X2Y2.
Those center coordinates.
So we are just want to get the center coordinates and the width and the height of the bounding boxes.
Okay, so.
Using this function, we just pass the coordinate coordinates and get the center coordinates, X, y,
z and width and height of the bounding box.
So now we are using a deepsort update function that is, and this function is called the update function.
The bot takes all the texture and just add the identities because he like you wallaby it will do the
detections.
Deepsort will only assign a unique ID to each of the object and draw the trails.
So dinshaw short, just assign unique ID so these IDs are not incremental.
So what does it means that.
Like you can see that 87, 68, 53, 67 like it is not 87, 88, 89, 90.
It assigns a ID randomly so it can assign a one ID to this object to 1011.
So it is not incremental form.
Okay.
Well, I think this is all from the code.
I hope you have understood this.
In the meanwhile, our script has been dropped, so let me show the output demo video which will be
saved.
Um, in this folder.
Let me go to this folder.
But what was the name of the project?
I think.
Rectangularly eared.
And then we will go to your deepsort alternatives.
Yolo.
We ate.
It runs with the tree.
And here is our output video.
Let's see the results so you can see that we have a unique ID with each of the object.
Thus we have a tracking trace.
We have detecting the object.
We are able to track the object.
Well, the model is working quite fine.
We are able to detect, track and assign a unique ID to each of the object.
So this is all from this video tutorial.
I hope you have learned a lot from it.
See you all in the next video tutorial.
Till then, bye bye.