Hi guys.
So we are ready to develop the installing algorithm and the very first date that we'll be developing
will be the localization states now inside the state.
The primary goal is to localize our robot inside the robot.
For this we have the Salvador part where we have the missiles asylum function that will be used iteratively
on all of those stages once they have been developed.
Now, because we are working on the localization states and we haven't developed it yet.
So first we need to develop it before calling the asylum asylum function.
So let's go inside the maze port and create a new that we named as bot localization.
But firefighters are inside it.
The very first thing that we need to do is to simply import the two most important libraries that are
civil to by library.
Now CV two is to import the computer vision algorithms, and numpy is great for creating an empire.
Then we create the class of mortal collider that will perform the actual task of localizing a robot.
The very first function that we'll be writing inside this class will be the initialization function.
So that can be done by simply writing definition in its first argument of self.
And then we define is instance variable.
The first instance variable is a boolean variable that simply tracks whether we have extracted the background
or not.
Because this is the Initialize and states, we haven't extracted it.
So we set this to false and then we create a placeholder for keeping our background model.
Once we have expected this, so we initialize this to an empty list.
Once you've done this, you move on to the next function.
That will be our main function for this.
Localization states that will be known as localized board function.
This takes in the input argument of self and the second argument of grand frame.
The third argument of frame to display.
Now frame display is be simply used to display the board that has been localized on the frame.
Now we defined this part of a function that is that we check whether we have extracted or whether we
have performed the full step of our location state.
That is whether we have extracted the background or not.
So that can be done by simply writing, if not.
So now it is background obstructed or not.
Now, if it has not been extracted, then we simply call the function that extracts the background.
So there is no one to extract background in providing the current frame as an input.
And then we simply after that we set the boolean to true because the background has been extracted successfully.
Now this means the frame to implement this particular function of extract background.
Now let's do that.
So we go up right here.
The definition.
Extract background.
Providing the first argument of self and the second argument of frame.
Now inside it, the very first thing that you need to do is to simply extract the mass of all the region
of interest that are present inside the frame.
So extract the most of all the region of interest.
Now, what are those?
Region of interest?
There's the Miz and there's the car.
So how do we extract from the video we are going to extract them is using as detection algorithm.
Now edges are only present for those objects that have sharp features around them.
So let's do that.
So the way we do that is using the detecting algorithm and the cutting edge detection algorithm first
requires a grayscale image.
So we need to convert our RGV or BTR color frame to a grayscale image.
And that can be done by simply writing simple to convert color, providing the frame as the input.
And Stevie to color.
BGR.
To Ray as the court.
This gives us the output of break and then for the next we simply need to pass this grayscale to the
cancer detection algorithm.
Now can you takes an image of grayscale, then we provided the range of 50 to 150.
Now this range provides adequate edges if we want.
Then for the next argument for edges, we provide none.
And for upward your size, we provide three.
Now, after applying the scanning detection algorithm to get edges as output, once we have the edges,
we can now use the open CV function of fine contours to find the contours of all those regions of interest.
So we use the function of fine controls that takes input, argument, image that is ideal for the mode
we provide in CV to return external, to simply get the external contours of all those objects and for
the method.
We provide in Syria to saint approximation none now because this function of plus two or two gives to
outputs and the only one, the first output that is the control.
So we need to write here zero.
So this will give us the output of controls.
And once we have the controls, we can simply now loop over these controls and simply draw them on an
empty placeholder.
Now, at the moment, we have not defined that placeholder.
So let's define a placeholder because we were expecting the region of interest for the most intimate
region of interest.
Must we use the function of number zero?
We need to provide a ship that can be taken from the shape of the frame.
So you provide the shape of the frame, except that you won't want the dare type of the frame because
the airframe was of the dirty type.
BGR and we want to get it out.
There it is, Chris.
That is off the mark.
So we tried d-type.
Of number know and it.
So once you've done this, we have the region of interest miles from the place order created.
Now we can simply go all in or simply call it a loop.
It loops over all those contours and gives us the ideas of these controls.
And that can be done for.
And by calling enumerate function, depositing the controls as the iterable, and then simply calling
the function of CB to draw controls and draw controls takes in the input argument of the image, which
is our placeholder that we have created just now.
That is a region of interest, Miles.
For the controls group Paulson controls and for an index forcing iterable that is idea and for the color
three passing to 55 and for thickness we write negative one receives to fill all those controls.
Now what this function will do was simply draw all of the mask of all those regional interests that
we are just detected now.
Once you've done this, you've completed the step of expecting the mask of all the region of interest
for the next step.
What we need to do is to simply remove the guard from this region of interest, because although it
is a region of interest, but this is not part of the background, the guard is the part of a foreground.
So to get the background or to extract the beginning, we need to remove the car from the background.
And the way we can do this is simply leveraging the information that God is the smallest object on the
scene.
So we need to find the object this one has gone to.
So for this, we need to use a function that we have created inside the utilities folder, the utilities
file that we have created earlier.
So let's import that file and use the function for the smallest object.
So we need to import it right here.
Import not you to or from from go to utilities to import the returns.
Smallest object.
So we go right here and simply call the function that we have just important files in the controls,
which gives us the output of index of the smallest title.
So we named that as minimum control index.
And once we have this, we can now use this index and to eliminate the car from all those region of
interest that we have extracted earlier.
So we don't want to overwrite over the region of interest lost, but we need to create a new placeholder.
So we name that placeholder as a region of interest with no card.
So this is a region of interest, no card models, and this will be a copy of the Region of Interest
Master.
Just in a different place.
Once we have this, we check whether the minimum control index was legit or not.
So if it was legit, it will not be equal to negative one.
Meaning we have found this one as control.
Which is actually the car.
So.
If it is not equal to negative one, then what we do to you to see to draw control function.
To draw over the image that is a region of interest.
No car must provide in the controls as the controls for the index.
They will not provide the minimum contour index because we have to do it on the models of our car and
because we are removing the car from all the region of interest, we need to write here zero as the
color.
So zero will simply remove the car as a region of interest.
And because we want to complete the complete car, not just as we need to.
Right.
Has negative one.
Once you've done this, we have removed the car from our region of interest.
Now, we also require the models of the car separately from me to create a new placeholder.
So we create a new class folder that we name as car models and then draw that over the car models.
The mean control index by providing the color of 25 and then diluting it by drawing with a thickness
of three.
Once we have done that, we can move on to the next step.
Now, because we have the car models will be required area that does not include the car or does not
include the car region.
So that can be done by simply using the function bitwise not not bitwise not simply inverse what ever
was one zero and 0 to 1.
So we provide in the car models to get the opposite of the car models for the region.
That does not include the car.
So that can be done by simply writing a car list or not.
Car must.
This means that we have the region that does not include the car.
So we can use this mosque.
It does not include the car to extract the frame.
That does not include the car.
So we can now extract the frame where the car is removed by simply using the bitwise in.
Which wasn't providing the frame a source one and source two.
And then for the mask and now providing the not guard mask.
Now this simply says only get those region of the frame.
That does not include the car.
And this becomes the frame the car remote now because we have the frame with the car removed, we have
an empty region inside the frame.
So we cannot say that this is the background as of yet.
We have to fill that empty region with the ground replica.
So for the replica we simply take the color of the frame or the top left corner of the frame removed.
So let's do that.
So again, the frame removed and it's top left corner.
First.
And named that as the best color on this fiscal.
It would be used to create the brown replica that would fill in the empty space inside the area where
the car was removed from.
So this cleared the ground replica
and that can be created by simply using the number function of ones like.
This takes an input argument of frame because we want a colour image the same size of the frame.
But we want to multiply this with a base color to get the ground replica.
Now that we have the ground replica, we can use this information to get only the ground region where
the car was present and then fill it with the frame car.
And that can be done by simply using a few of the bitwise and where we get the ground replica only in
the region where the car was present and then simply or in that region with the frame removed.
This gives us the background model.
And now we have extracted the background model.
The main task of this extracted background function.
The main goal of this particular localized state was to localize the robot inside the mix.
So we need.
So we want to localize our robot relative to the Miz.
So to get to the relative home is really quite a frame of reference.
So we need to get that frame of reference or get that means cropped out from the whole frame.
So we require some sort of boundary enclosing that means.
So let's do that.
Okay.
So for that, we first need to create a function that encloses all the region of the means and the car
inside of it.
So we create a function that does exactly that.
So that function would be known as return region of interest bounding.
So we provide in the region of interest models and control, and this will give us the whole that encloses
all of the mess inside of it.
So we simply now call that particular function by simply writing self interest, bounding or providing
the region of interest and the controls.
This gives us the whole that encloses all of those region of interest and includes the means in the
car.
And once we have the whole, we simply use the bounding rect to get the rectangle that bounds, that
hope.
This gives us the starting rule, starting column width and the height of the bounding rectangle.
Once we have that, we use that region to crop out the mes from the whole frame.
So we get the region of interest with no corners and use a crop of the maize from that particular region
or that particular mass.
Once we have the mis cropped out, we can now.
Get them is occupancy grade where the car contributes by simply inverting the is.
And the reason you're inverting them is, is because that region with the males was present or the males
boys were present is directly opposite to the region where the card controls from.
So right now we have the was detected, but we want the opposite area to the wall.
So we need to use CV to bitwise not so we use that which was not to get the occupancy group.
This provides this to me documents secret.
Now this gives us the maze occupancy, but we also want it to be a specific frame of reference.
So to maintain the consistency in the frame of reference, we always point entry point of a maze on
the top.
So because the entry point of the maze of this maze is on the right.
So we need to rotate it counterclockwise.
So let's take it to get the final maze occupancy grid.
And that can be done by switch to rotate, providing the maze occupancy rate.
And for the code we provided Siri to rotate.
Counterclockwise, 90 degrees.
And this gives us so of May's occupancy.
Great.
Now.
We also need to create this.
Inside the function for declared distance in function.
By writing these documents include initializing it with an empty list.
Now this will the second output that we required, and that also become crucial in the next stage for
the mapping stage.
Now, once we have done this, all that is left for us to do it will simply update the parameters that
we used to get this frame of reference.
That is our maze occupancy grid.
That includes the cropping barometers and the rotation parameters.
So let's update those parameters.
So first we need to do is create a few more instance variables.
This includes the growth parameters X, Y and row and column of the original frame and the rotation
angle that that was used to rotate out of our frame to get the maze entry on the top.
Now we need to create a new function that would be used to update the frame of reference parameters.
So we use this particular function by calling it right below here.
So we call that particular function of or update frame of reference provided stock roll start column
width and height and angle to which on which we rotated our frame because we rotated to 90 degrees,
we provided 90.
And this simply updates the reference parameters that can be used later on when you want to reference
on bear on the original frame B crop to get our frame of reference.
Now finally, simply display all the intermediate stages and the final stage that we got by implementing
the function of the extract background.
This includes regional frame, Christmas frame column, ground replica burger model and main documents
secret.
This last two outputs are the final outputs of this particular function of extract background.
So the background model will be used to use to get the foreground using the background subtraction and
the mixed occupancy gear will simply be used for the next stage of mapping when we want to identify
which region the caller controls.
So now let's run this and see whether we were able to correctly implement the first step that is background
extracted for the background subtraction task of local and state.
So let's do this now before we can run this project.
We still need to import it inside the main Salvador part.
So let's go over the missile part.
We're right at the start and import from our board localization import the class that is known as localizing.
Then we simply create the object of this class.
Four.
And the object for the missile over and give it the same name as a class object that is bought.
Localize it.
We initialize this to the localize that object.
So once we have the object created for this missile class, we can use this object inside the Mir solving
function.
We're writing.
Silver that bought localized, localized sport.
And that takes an argument of grand frame, which is basically sales or set view or the satellite view
that was being provided to us from the top down camera.
Looking on the means for the frame display, we need to create a new variable or placeholder that we
name as frame display.
And this is simply a copy of self thoughts that you.
Now to provide this information, we can now simply build this project and see whether the full step
of the localization states is working correctly or not.
So we move inside the terminal.
Will the project.
Launch a simulation.
So when this once the simulation starts up, we can now run the MIT solver.
And as you can see, it has grown successfully and it has given me the five outputs.
Let's see the original output.
That was a satellite view.
As you can see, we have the mess looked down upon from the satellite and we have the we have the board.
Right.
And this miss entry point of the miss.
Then for the first output that we got was the region of interest lost.
Now these models are all of the region of interest that represent inside the frame that included the
Miss Walls and the car.
Now, what we did was simply extracted the car mask and simply removed it.
From our mainframe.
And what we got as an output was this frame car removed, which included the frame, but the car was
removed altogether.
But this left an empty space right here.
So we fill that space with a ground replica.
So the ground replica was simply of the same color of the as of the bag known for the mace.
So we simply filled the space that was left empty.
And the final output, that record was the background model which included the mace inside the frame.
But the foreground that was the car was simply removed.
Once we have the background model, the first output, we can now move on to the second output that
will see on this particular step of the block localization states.
The second output was the occupancy grade.
And as you can see, not only the maize has been dropped out to get the frame of reference, it has
also been rotated to get the maize entry right on top.
So this will be our frame of reference on which the board will be localised.
So the port is on right here on the maize entry.
It is basically on the top right position and can be interpreted as the frame.
The columns of the frame are a few rows, a few columns before the next column of this particular frame.
So this is our frame of reference.
The second output of the extract background function for our local and state.
Now let's implement the second step to use the to use this full background model that we have just expected
to extract the foreground that is the crown.
In the next few frames till then.
Thank you.
And by.