Hi and welcome back to Chapter 23, where we take a look at background and foreground subtraction,
so open up that notebook and let me explain to you what we're doing here.
So what we're going to do, we're going to try a few different methods of background subtraction.
No background subtraction is what do you expect?
Remember this and all images?
Most images, I should say, is a foreground subject, which is an object that's probably being photographed
or focused on.
And then there's a background, which is the stuff that's behind it, or it doesn't even have to be
behind it to just around it.
So what if we wanted to just remove the background?
So get this black mask over it?
I'm going to keep the white area.
That's what background so production is, and then foreground subtraction is the opposite.
It's where we actually want to remove this boat and you only get the background here.
So open TV has a few background subtraction algorithms implemented, which are quite good.
So we'll take a look at those here.
So let's lodo images and download the little libraries, I should say.
So let's move.
Move ahead, and we're going to use this demo video, which is a walking clip of pedestrians, which
is which is famous in computer vision, which is why I use it.
It's always good to give those previous researchers some credits for when they use these videos.
That's this video here.
You would have recalls seeing it before.
So let's take a look at what backroom subtraction is.
So actually did mention it to you before, but if you want, you can read this in more detail if you
want to get it.
And algorithm, we're going to try first.
It's called a Gaussian mixture based background and foreground segmentation algorithm.
I'm going to explain this here because it's quite complicated.
And here's a link to the research paper.
If you want to see it, see it for yourself.
What I'm going to show you, though, is how we implemented it here in open TV.
So you remember how we actually created this video, which in the video, right?
So which is the output?
So we do that here again.
We load the video and then we create the outputs here.
So this is the part that does a background subtraction.
By subtraction, it's KVUE, Dot, B.G. segment M Dot, Create Background, Subtract MLG, which stands
for that Gaussian mixture got a mixture of Gaussians actually is what sounds sensible.
And then we just go to the loop with which each frame of the video and we get, we extract this this
bit.
Sorry, then extract it.
This gives us the we apply this to the frame to get the foreground mask using the airplane mounted,
and we write that to filter on this, which is the output to the the output video we created here.
So let's run this what we're doing here, we're actually printing out or displaying.
I should see all the images here in the video, so we don't actually have to watch the video in the
end.
We can see it live here frame by frame.
So you can see these are the pedestrians that we're walking here, and you can see that it actually
is working fairly well.
You're getting the pedestrians and everything else does not.
Moving is black, so it is working as expected.
However, it could be better because as you can see, there are a lot more pedestrians here than it's
letting on.
So we can just go back here and now let's try something called the improved adaptive Yorkshire mixed
model for background subtraction.
So we're going to do the same thing again where we display all the outputs and immediately you can see
that it actually is working a lot better.
You can see that the pedestrians are much more defined.
So we have a good foreground mass going here.
This is the background has been eliminated in these images.
So if you wanted to just extract the foreground, you can use a Bitwise operation to do that, which
is a good exercise for you to do.
Actually, a good exercise for you to do is to do the Bitwise operation and on the one, the original
image and write it to an output image and then display that video and call up to previous quarters in
all the notebooks.
You can check it out and everything should be simple.
So let's take a look at foreground subtraction.
Now, moving on to foreground subtraction of foreground subtraction is actually done a bit differently.
It's done a bit more mathematically now.
So what we do?
We load a video and create a video, right?
As well, just just in case you want to see the saved output.
But we use this function called V to accumulate tweeted.
This function allows us to basically, as depicted as a video sequence goes, allows us to store values
of that.
So that accumulates values and then we get the average as well, which is what we specify here.
So we get the average value from the frame.
So it's a way to actually kind of keep track of what the background is.
So then what we do, we use this?
The function here called C-V2X convert skills absolute.
This actually scales and calculates the absolute values and then gives us the result as an eight bit
unit, unsigned end, and that's how we mathematically get the background.
So let's run this and we can see the outputs as they come out.
Let's zoom in a bit and you can see as good as it progresses to the bottom.
Sorry for the haphazard scrolling.
Just because I'm zoomed in, it looks a bit extreme.
This is a disappearing background.
As you can see, it's not that apparent in these images.
However, it accumulates over time, so the longer we leave this, the more it'll accumulate.
So what we can do?
We can show the last image from this video and let's look at this here and you can see.
So it's not the best method, and it should play around with the values to see if we can get it to be
better because I'm sure it can be.
But you can see this guy, they look a bit invisible, these pedestrians.
So it kind of works, but they're better at measurement methods around it.
Let's go back to this, so let's take a look at another background subtraction method called it using
the can a method.
So it's just basically the similarly like we did before.
However, we actually using creating Akinwale from the TV to get structure and element, which I'm actually
not too familiar with what this does.
However, we use the cleaner for this background subtraction, subtract, subtracted key and admitted.
So we can actually apply this now in the frame.
So you go to you create this.
This is the foreground background here.
We used apply method inside of that object applied to the frame and get the foreground must then apply
the morphology x, which is, you know, using function with the kernel that we defined and then we
get the output.
So let's take a look at how this looks.
So you can see this again works pretty well.
You can see the actual people are very well defined.
So this is perhaps the best background removal method we've come across so far.
So that concludes this lesson on background foreground subtraction.
The next lesson is going to be on motion tracking with the mean shifts and camshaft algorithms.
So stay tuned for that.
And thank you.