Hello everyone.
This lecture presents an introduction to multiple object tracking using sort and deep sort algorithm.
So stock sort stands for simple online real time tracking both sort and deep sort are multiple object
tracking algorithms.
Deep sort is an extension of sort algorithm.
Basically, the authors of the sort algorithm have proposed that deep sort algorithm to address the
issues in the sort algorithm.
In this lecture, we will see what are the issues in the sorted rhythm and how does deep Sort solve
the issues in the sort algorithm?
In this lecture, we will start from the scratch and will try to explain you.
What is basically tracking is why we need an object tracker.
Then we will also look at the applications of object tracking types of trackers and so on.
So let's get started.
So following are the objectives.
Or you can say that we have divided this whole lecture into eight different parts.
So in the first part we will see Introduction to tracking.
In the second part, we will see why we need an object tracker.
In the third part I will discuss the applications of tracking.
In the fourth part we will see the types of trackers we have, which include single object tracker and
multiple object tracker.
In the fifth part we will see what is Deepsort algorithm.
In the sixth part we will discuss about sort algorithm, which is simple online real time tracking algorithm.
In the seventh part we will see the issues in the SORT algorithm and in the eighth part we will see
how sort deep sort algorithm solve these issues.
So let's get started with this first part.
So this slide presents an introduction to tracking.
So what?
Basically, object tracking is So object tracking is basically a method to track the detected object
throughout the frames using their spatial and temporal features.
So what basically we do in tracking so in tracking, we get the initial set of detections from the object
detection algorithm, which can be YOLO, V7 or YOLO V8.
In the next step, we assign the detected objects a unique ID and then track them throughout the frames
of the video feed while maintaining the assigned unique ID.
So object tracking algorithm assigns a unique ID randomly.
Basically, it can be any random number.
Unique ID can be any random number.
It should not be in the form of order.
So like the first object can be assigned the ID of 30.
The second object can be assigned an ID of 35, 40 or 100.
Okay.
So in the let's see what basically tracking, what process the tracking, what So tracking can be considered
a two step process.
In the first step, we do the detection and localization of the object using any type of object detector
algorithm which can be Yolov7, YOLO, V8 or YOLO.
R So after doing detections in the second step using the motion predictor, we predict the future of
the object using its past information.
So this is how we implement tracking.
And tracking is basically a two step process.
Let's move towards the next slide.
Let's see why we need an object tracker.
When I start reading about object tracking, I was thinking why we need an object tracker.
Why can't we use only object detector?
Let's discuss this with an example.
So why can't we use an object detector only?
So in a video feed, for example, we are detecting the car whenever the car gets occluded or overlapped
by something.
For example, with a truck, the detector will fail.
But if we have a tracker with it, we will be able to predict the future motion and as well as track
the car by assigning a unique ID.
So in object tracking, we assign a unique ID to each of the object.
We want to track and maintain the object that ID till the object is in that frame.
Let's explore the applications of tracking.
Let's look at the first application, which is traffic monitoring, so trackers can be used to monitor
traffic and track vehicles on the road.
They can be used to judge traffic, detect violations and many more.
Other applications of tracking are there, which includes automatic number, plate recognition and so
on.
So we can also use object trackers in sports as well.
So trackers can be used in sports like ball tracking, player detecting or shuttlecock trick tracking
in case of badminton.
This is in turn can be used to detect false scores and many more in the game.
So in game we have many applications object tracking which include the in basketball, we can count
the number of scores.
We can also see if there is a foul in football or we can also use a to track ball in case of cricket
or in case of football.
We can also track the football or in case of badminton, we can also track the shuttlecock.
So in case of multicam surveillance, in tracking, multicam surveillance can also be applied in this,
the core idea is re-identification.
For example, if a person is being tracked in one camera with an ID and the person goes out of the frame
and comes back in another camera, then the person will retain the same ID that they had previously.
This application can help in identifying the object that appear in a different camera and can also be
used in intrusion detection.
Let's explore the types of trackers.
So there are two types of tracker.
Single object tracker and multiple object tracker.
In single object tracker, we only track a single object no matter how many objects are present in the
frame.
Single object detectors trackers are actually very fast.
Some of the single object trackers are cshd and many more as well, which are built using the computer
vision.
So in multiple object tracker, we can track multiple objects present in a frame at at the same time
even of different classes while maintaining a high speed.
So multiple object trackers have proved have proved to be more accurate.
Some of the commonly used multiple object trackers are deepsort sort and central track algorithms.
Hi, guys.
Let's look.
What is Deepsort algorithm?
So Deepsort is a computer vision algorithm used to track the objects while assigning each of the tracked
object a unique ID.
So Deepsort is an extension of sort algorithm.
Deepsort was introduced by the authors of sort algorithm to address the issues in the SORT algorithm.
So Deepsort algorithm introduces deep learning into the sort algorithm by adding appearance description
descriptor to reduce the identity switches and hence making the tracking more efficient and more accurate.
Guys.
As I told you, Deepsort introduces deep learning into the SORT algorithm by adding appearance descriptor
so you can see here.
Here we have added appearance descriptor into the SORT algorithm.
So this becomes deep sort algorithm.
So appearance descriptor helps us to reduce the identity switches and hence make the tracking more efficient.
So.
This note.
The conclusion is Deepsort basically introduces deep learning into the SORT algorithm by adding appearance
descriptors, which helps to reduce the identity switches and hence makes the tracking more efficient.
So here you can see that in the SORT algorithm.
We have added a previous descriptor over here, over here.
So this becomes now our deepsort algorithm because deep learning is introduced into the SORT algorithm.
Let's look at this sort algorithm.
So sort is an approach to object tracking where Kalman filters and Hungarian algorithm are used to track
objects.
Sort consists of four components which are as follows.
So the first component is detection.
In the first step, the detection of all the objects which are basically needed to be tracked is done
using Yolov5, YOLO, V7 or YOLO V8.
So after doing detection in the next step, estimation is done.
So in estimation step, the detections are passed from the current frame to the next frame to estimate
the position of the target in the next frame using Gaussian distribution and constant velocity model.
The acoustic estimation is done using the Kalman filter.
The third step is data association.
So in that association, the cost matrix is basically computed as the intersection over union distance
between each detection and all predicted bounding boxes from the existing target.
In the fourth step, the creation and deletion of track identities is done, so when the object enters
a frame, it is being assigned a unique ID, and when the object leaves the frame, the unique ID is
removed from the list.
Naught algorithm performs very well in terms of tracking precision and accuracy.
But there are issues in the SORT algorithm.
The sort algorithm fails in case of occlusion and different viewpoints.
And also, despite the effectiveness of Kalman filter, it returns a relatively higher number of identity
switches.
So these are the two issues in the SORT algorithm.
So the authors of the sort algorithm propose the sort algorithm to address these two issues.
But we have seen that there are two issues in the sort algorithm.
First one is sort algorithm fail in case of allusion.
And the second one is that sort algorithm returns a higher number of identity switches.
So the authors of the sort algorithm propose deep sort to address these issues.
So in deep sort, another distance metric is introduced based on the appearance of the object, which
is the appearance feature vector or deep appearance descriptor.
So Deep Sort uses a better association metric which combines both motion and appearance descriptor.
So deep sort can be defined as a tracking algorithm which track object not only based on the velocity
and motion of the object, but also based on the appearance of the object.
The following is the block diagram of Deepsort algorithm.
Deepsort introduces deep learning into the SORT algorithm by adding a adding appearance descriptor,
which you can see over here.
So as the appearance descriptor reduces the identity switches and hence making the tracking very much
more efficient.
So this is all from this lecture.
See you in the next lecture in which we will implement Deepsort tracking in Google CoLab, and we will
also implement sort algorithm in the upcoming lectures as well.
So till then, bye bye.