So now let's move on to section 25, where we take a look at object tracking with optical flow.
So open that notebook and really have it loaded here.
This notebook has some good blocks that do take a while to run.
So I've run, I've run all of them before.
This lesson started before the recording, so we don't have to wait for them to complete.
But you can run them and wait for them as you go along this lesson.
So firstly, little libraries and images and videos that we're using in this case.
And let's move on to the first part of optical flow.
So optical flow is another tracking algorithm, and it's a much better tracking algorithm than the previous
mean shift and camshaft.
We looked at what it does.
It looks for the apparent flow or movement or direction of an object that's moving in an image and between
consecutive frames.
And then basically tracks that with the 2D vector field, where the feature vector represents the displacement
of the movement of points from frame to frame.
So you can see in this image here, as this Red Point moves here, it generates this vector because
you can see from point one to point to it moved here.
And likewise and likewise so you can see it's projecting this vector that is indicative of the direction
of this object.
So you can see it here.
All right.
So it's quite useful and it's quite a good algorithm.
I remember using this in university during a Masters project while I was helping a friend with a master's
project where she was tracking ants in a frame that was moving.
It was moving story, obviously from a top-down perspective.
And she used what we use optical flow to track the movement of the end because they were trying to figure
out what is the algorithm that the ants use to navigate.
So it was pretty cool.
So let's go on to how we implement optical flow using open TV.
So because we're losing a video, we're going to create a video writing functions here.
And then what we do, we used a she tomasi quantum detection algorithm.
That's what's used in.
Actually, don't mention it here, but that's what's used in optical flow.
At least one of the methods we can use in optical flow to identify the points that we need to track.
So we use that there.
And then this is the parameters that we establish with it.
So you can always measure only these parameters are it's highly configurable.
This algorithm and we declared and Lucas Canada, because that's the person with a group of persons.
I'm not even sure who developed optical flow.
We just create this here with these parameters as well, which is window size max level, which indicates
the amount of pyramids.
So that's we can.
That's a scaling tool that open to the user so we can look at two, we can make it look at different
skills and more robust to smaller or larger objects that we want to track.
And then what we do?
We use this function.
Here's this random function to create some random colors that will create the trails in the image so
that each trail we can distinctly see which each real is and then find the initial coin is here.
So using the CB2 function called Good Features to Track, which is appropriately named.
And this gives us the corners.
Oops.
Let's go back down to the code.
Sorry.
This gives us the coin is that we are going to initial coin as we're going to use to set our tracking
points and then we the mask for the drawing purposes right here.
So you know how it grew previously in open TBE world, we actually do need it for other functions when
we were integrating and demonstrating different functions.
So now let's look at the optical flow code here.
So you can see we just thought this loop would read frame by frame can fit it into a grayscale frame,
and we input the previous grayscale frame, which is the first frame from the image.
With the current frame here, which is now a frame grid with the initial coin is here, and then we
just send this non, which indicates that things are using different parameters for something in the
optical flow function.
And then we send Lucas Canady parameters as well.
So then all we do is select and store all the good points that we want to use here.
So it's Tuttle's equals to and one gives us two points that we want to keep.
And then we just draw tracks using these functions here.
So I'm not going to go into this code in too much detail because it can get a bit complex, especially
for beginners, because you don't know what its function is and enumerate and those things.
But basically what we're doing, we're just iterating through these values here and putting in new and
all in each one.
That's why we use Zip to do it for a list like this or a set like this.
And to this actually, which I should say and we use Laravel to flatten these as well, just give us
two values here and then we just draw the lines and.
Stuff there, and then we just add that frame so we can actually upload it to a video writer.
And then we did the previous values, so we know have a new previous great PBS corner, which is a which
is the current one for a loop.
And then when the video finishes, we break.
So we end that.
So then so this takes about 11 10 seconds to run.
Conversion takes about 18 seconds to run the bubble.
Run all of this already and displaying the video, it takes a little while to run for some reason,
but we can display it here, and you can see this is pretty cool.
You can see as a person walks, you can see the trails behind them and the trails.
And sometimes when it loses that track, the track of that person is always super reliable.
But you can adjust the sensitivity with the parameters, and you can see this is how it works.
You know, it's using the good features, a track which just finds corners.
That's why it is like these interesting areas here, which are probably corner points.
You can see one here.
Lampposts is a bunch over here by this parked vehicle.
So all of those are points that we initialize, but then weren't moving, so they weren't tracked.
And the reason we have different colors is because we used that random color function to just go up
here just to generate random colors between groups between zero and 255 here as a vertical ranges.
So let's go back down.
And now let's take a look at another implementation of optical flow.
This one is called dense optical flow.
So dense optical flow uses a slightly different method, which is illustrated in the gun of funny box
algorithm.
Sorry, but this peroxidation, which is explained here.
Basically, it's a bit more complicated, and I'm not going to go into the actual explanation of how
it works.
However, this is like a good little recap this line that shows you how the original optical flow to
Lucas Canady implementation worked by using the sheet Thomas Tomasi algorithm for getting the corners.
So what we do here and in this video, what we're going to do, which is a little bit different, is
that we're going to basically make the background black and track and only show it hitting the objects
we want to track and direction is going to correspond to Hue.
Remember, Hue is color and magnitude of which is a brightness of it is going to correspond to the speed.
All right.
So you can take a look at a code.
You can see it's it's similar to the code we previously looked at.
We create the video, right?
So he'll get the first frame, establish these things.
We create a blank.
You just feel that we're going to use here, make it all white initially.
And then what we do, we create the optical flow from the back.
So the parameters and a lot of parameters in this algorithm, which personally, I don't know what they
do if it's interesting to you, most of it does affect the sensitivity of the tracker.
So you can take a look at this function in the open, see this documentation and see what it does.
That's something I encourage you to do.
There are tons of open TV functions, and it's impossible for one person to know all.
However, when you're checking your you for your application purposes, you will need to configure some
of these algorithms.
So definitely, if it's something that's interest to you that you're working on, definitely dig up
in documentation and take a look and see what's going on.
So what we do here, we use a flow to calculate the magnitude of which is the speed and angle of the
motion.
And then we use a values to reflect the speed at an angle again.
So that's how we use different colored color combos and just be here.
You can see it's indexed, right like that there, which hue and value.
We ignore saturation in this case, because that doesn't change the visibility to us.
A highly saturated on saturated color will be a bit more difficult to see than the hue and brightness.
So I've run this already.
This one takes a little bit longer to run next.
With just over a minute, the conversion takes about 15 seconds and then we can display it here.
So let's take a look and immediately see, it's a black background, which we created initially.
We remove we created a white background, but then we set those values afterward to zero where it doesn't
have motion.
So let's take a look at the output.
So you can see this looks pretty cool where you can see when someone is moving the left, you can see
dark blue or pinkish.
And when someone is moving right, they're yellow or reddish.
So you can definitely see that the color corresponds to the direction speed, the faster something is
moving to brighter.
Look, you can see it here when it's close to the camera.
The apparent movement does appear to be more as than someone put it away.
So that's why brighter objects because objects will appear brighter.
So that is the implementation of the dense optical flow tracker.
These are very useful.
Is that can be used for many different applications, like tracking vehicles, tracking people, tracking
animals, so feel free to use them for your application purposes.
So that concludes object tracking windows policy with other items.
Next, we're going to make a very simple object tracking by color.
So stay tuned for that.