Welcome to the third project.
In this project, you will create your own Ross service interface and implement it into your code using
what you learned from the previous lecture.
For this project, you will make two nodes one node which acts like a client, which sends a request
of an angle to a robot to turn a camera and another node which takes in the request and sends back a
response, which will be an image the robot takes after moving the camera to the angle specified.
Now, since we do not have a real robot and haven't learned about making robots in simulation yet,
we're just going to have some pre taken images which are named by an angle and return one of these images
based on the request.
These images are available in the resources folder or you can create your own.
It may be helpful to spend some time to think about what message types would be most ideal for our request
and response.
So let's break down what this project will look like.
For this project, we will create two nodes a service client node and a service server node.
The service client node will send a service request of how many degrees to turn the camera to the service
server node.
The server will then return the pre taken image data back to the client.
Then as an added feature, see if you can make your service client.
Display the image for the user to see.
In order to complete this task, I will use the OpenCV Library, which is a commonly used computer vision
library.
I know we haven't explicitly gone over OpenCV in this course, so I'll quickly go over some useful functionality
relevant to this project.
In order to use OpenCV, you must install it.
We will also utilize the complementary CV bridge package to help us convert between ROS and OpenCV data
types when working with images.
We need to also make sure we include dependencies for OpenCV in our package XML file as well as our
C list text file.
To import OpenCV in your python code simply call import CV to.
Then we can also import the CV bridge object from the CV bridge ROS package.
The common utility when using OpenCV is to open an image file on your computer.
We can do this with the image read function shorthanded as I am read, followed by a string with the
file location.
You want to display an image, you can use the image show function which will open a window with the
corresponding image you want to display.
Then we can use the weight key function with a value of zero to signal that the window should stay open
until we close it with the enter key on our keyboard.
Then lastly, you can also have your code ensure all OpenCV display windows are closed when finish running
your script by calling destroy all windows.
So that's just about covers all you need to know about OpenCV.
So now let's jump into CV Bridge.
Once you've written the image, you may want to convert it to a ROS image message from the sensor messages
ROS package.
You can do this by calling the CV to to image a message method of the CV bridge object, then to convert
a ROS image message back to an open CV data type, you can use the image of message two CV two method.
Also a helpful reminder on how to use the image interface type in your custom service.
Here's an example of how to use types from a different Ross package and how to include these ROS packages
as dependencies within your CE.
Make list text file.
All right.
With that OpenCV prep out of the way, it's time for you to try this project out on your own.
So go ahead and download the sample images from the resources section of this lecture.
Then make your custom service, which will allow you to request a degree for the camera to turn, which
responds with an image.
Then, with the help of OpenCV, create a service server which will take the request and return an image
from the sample images, as well as a service client which will be used to make the request.
And as an optional challenge, display the image returned in an open CV window.
With that, pause the video and give yourself an hour or so to try and complete as much of this project
as possible on your own.
All right.
How'd it go?
I'm sure this was a hard project being that you had to use OpenCV, and we have not explicitly covered
it in this course, but this is exactly how it feels to be a robotics developer.
You may have also been able to achieve the same result with a different framework or your own custom
solution.
But as a developer, you always need to balance how to incorporate advanced functionality into your
robots as quickly as possible.
I will go ahead and walk through how I would approach this project.
In order to do this project, make sure you have downloaded the images from the resources section here.
I've gone ahead and downloaded it and unzipped it in my downloads folder, so I'll just go into my package
directory.
And copy the images into their.
All right.
Let's go ahead and close that.
And the other thing to note is I will be using the OpenCV Python module to utilize some of its image
reading, processing and display functions in order to install it.
You can open up a terminal and update your app repository and then install lib opencv dash dev and python
three dash OpenCV.
And so in this case, I've already gone ahead and installed them ahead of time.
But be sure to go through that installation process.
We can double check that this installation worked by starting Python three in terminal and importing
CV two.
And then we can go ahead and run the CV to DOT and then to underscores version and to underscores again,
note that if you have a similar or newer version, you should be fine.
But here we can confirm I'm able to import CV two in Python and here we can just reference what version
number we have so I can go ahead and exit this.
And clear the screen.
OC First thing I'm going to do is open up V's code and create a new custom SVG file.
I'll call this RV file Turn camera, RV.
For this project, I want to take in a number in degrees and return an image.
So I will use float 32 for my request and call the attribute degree turn then separate the request and
response section with three dashes and then for the response I will use the image message type from
the sensor messages library.
I will just call this response attribute image.
All right.
With that, I can save this file and head over to my make lists, text file, and we can go ahead and
add it to our ROS Idle generate interfaces Command.
Note that I am also using the image message from Sensor Messages library, so I need to be sure it'll
make that I also need that package and that it's a dependency for these ROS idle interfaces.
So I can do that with the dependencies keyword and followed by the name of the particular package that
it's depending on.
With that I can go ahead and save the file and we can head over and build the workspace.
All right, so I can go ahead and close this.
Open up our terminal, make sure we're sourced to our install setup file of our workspace.
And now I can run the Ross to interface list command, and I should be able to see our new turn camera
service interface within my package.
Great.
Now I can use this interface in my code, so I'll go ahead and make my service server node within my
scripts directory.
I'll just call it turn camera server.
As usual, I will just copy over the service server code that we created in the last lecture as a baseline.
And I'll go ahead and change the imported service to turn camera.
I'll also go ahead and change the class node service topic and callback function names to match the
context of this project.
And I'll go ahead and delete the current content we have in our callback function for our service.
Now, remember, for this project I will have five sample images which I'm going to choose from.
So I'll just make a note of these values up here in a variable called available angles.
So when I receive a request, the first thing I will do is check this list to which number is closest
to that of the number given by the user.
To make this a bit more organized, I will create another function called Get Image, which will help
with this processing by taking in the degree turn attribute of the request.
Then I can define this function below.
So I'll go ahead and calculate the closest angle by using the min python function and making a condensed
lambda function.
Basically, the min function will iterate the available angles list and I use the key argument to set
a function to find the min value of the given list.
In this case I take the absolute value of each available angle with the angle the user gives us.
Don't worry too much.
If you're not experienced with Lambda functions, you could have easily gotten the same results with
a simple for loop.
Then, just to make sure my code is working properly, I will print out the closest angle.
Then I can recreate the file name of the image by combining the string of fied version of the closest
angle in the available angles list and adding the dot PNG file extension.
In the spirit of keeping the code tidy and organized, I'll make a new function called Read an image
by file name which will be responsible for finding the image from within our images folder.
So now I'll create this function below.
Now, since we are working with reading and files, I will import the OSS Python module and use some
of its functions.
Now I'm going to want to find our image file dynamically rather than hardcoded in the file path in terms
of where it is on my computer.
So I will use the OSS path dur name function which will return the full file path of a file.
In this case I'll just use the underscore underscore, file underscore.
Underscore python variable.
Note that this will not lead us to the script folder if we are using the ROS to run command.
Instead, we will end up deep within the install folder of our workspace, which is where a copy of
our script is made when we run an build.
So I'll take this directory path string and index it for the install folder.
Now I can crop the string using this index so we only get the file path of our workspace.
Then I'll just take on that.
We want to go into Source Udemy raster package and then the images folder.
Make sure this path matches where you put the images within your package.
Then lastly, I will just tag on the file name, which will be the corresponding number pad.
Great.
Now that I have the full file location of the image, I can go ahead and read it, in which I will do
with OpenCV.
So I'll go ahead and import at the top of the script.
And since I'm here, I will actually go ahead and also import our CV bridge.
And so I'm going to need that in a bit.
This CV bridge is a ROS package, which you should have by default when we did our ROS installation
and we can actually check this by heading over to our terminal and I'll click the screen and do Ros
to package list.
And here we indeed see the CV Bridge package.
You can feel free to check out the OpenCV documentation if you want to learn more about the tools that
are available and how to use them.
For now, I will just head back down to my read in image function and we'll read in the image using
the CV to I am read function which will take in the file location.
Then we can go ahead and return that image.
Great.
So it returns the image to our get image function that we have here, which returns it to our original
callback that we have here called Return Image.
I now have to convert this image data into an actual ROS image message to load it into the response.
Well, the CV bridge CV to to image message function can do that for us automatically.
Then I can go ahead and take my response variable load the image attribute of it, since that is what
I named it in my RV file and set it equal to this image message variable.
Then I finish this function by returning the response.
That should be it for our service server.
So I'll go ahead and save the file and add it to our make list text file.
Now go ahead and save the file and rebuild our workspace to include this new turn camera server Hi file.
Now, let's go ahead and test this.
I'm going to make sure that I'm source do my setup file.
OC our server is running.
So now I'll open a new terminal tab.
And I'll source my workspace again.
And I can use the Ross to service Terminal Command to interact with my server.
In this case, I'll use Ross to service call to send a degree turn request of ten degrees.
And again, I'm hitting the tab key here for auto completion to help me fill in these commands.
OC We see a bunch of numbers flying across the terminal.
This is actually the raw image data from the image our server sent back.
And if we look above in our server terminal, we can see it calculated the closest image to be the 15th
degree image.
So it seems like our code is working just fine.
So now I'll go ahead and make a client that can actually take the image data returned by the server
and open a display window for us to see it.
So I'll head back over to V's code and create a new script which I'll call Turn camera client Dot Pi.
As usual, I'll go ahead and copy over the code that we created in the last lecture for our service
client as a baseline.
And I'll change the contents to match this project.
All right.
So we've gone ahead and created our turn camera client.
And I still have this self request call, which takes our turn camera service and creates a request
object, which we're going to be utilizing in this send request function.
Now, instead of an integer, I can go ahead and convert this over to a float and we will have to change
our request type to match what we have in our SRI file.
So my request has the attribute degree turn, so I'll go ahead and match that naming convention.
Then we'll go ahead and wait for the service server to be available.
We have our future object, which we will make our asynchronous call with our degree turn that we have
set up and spin until the future complete, until we return our result.
So the only thing I've got to change now is if I head down over here.
Change what our input is.
So in this case we will say enter a number in degrees to turn the camera.
Now, rather than just printing the result, we want to actually open a display window.
So within my class, I'm going to create a new function called display image.
And this is going to take in the image message that gets returned to us by our service server.
Now, in order to complete this, I'm going to need my OpenCV and CV Bridge imports.
So I'll go ahead and copy those from the previous file.
And so now we're going to use this bridge again to convert the image message to the CV to format.
Now I can use the open CV's show function to create an image display window.
I'll name the window, turn camera image and pass it in the image variable I just made.
Then I will use the CV to weight key function and set it to zero, which means that the window will
stay open until I press the enter key.
Then I can just make sure it closes all the open windows once we are done.
And it seems my tabs are not configured properly.
So just go ahead and fix this really quickly.
And that takes care of that error.
Then last thing I need to do is actually implement this function down in our main call.
So instead of just doing our server returned, we will take our result and pass it to our display image
function.
Great.
That's it for this node.
Let's go ahead and save it and go over to our Quick List text file and make sure we install this program.
So I'll save this.
I'll go ahead and rebuild the workspace.
All right.
We can head back to our terminal.
So our turn camera service server is still running.
So go ahead and clear the screen.
Make sure this workspace is source and run our turn camera client.
Here.
I get prompted how much I want to turn the camera angle.
I'll just set it to ten like we did earlier.
When we use the Ross to service, call command and terminal.
Oh, and we get an error here saying attribute error turn and camera response object has no attribute
encoding, so it seems like it's expecting an image, but it's getting our response object.
So that means I forgot to edit our callback function in turn camera client.
So here when we send the request, we get returned the entire result.
So this encompasses the entire result object.
So what I'm going to return instead is the image attribute of it, which is the attribute we named here.
So now it's actually returning that image message instead of the response object.
So if I go ahead and save that and we can go ahead and clear the screen and try rerunning this file.
I'll try my ten again.
And this time we get a display window which has the corresponding image for 15 degrees.
Since that was the closest value we had to ten.
Close this window, we can just go ahead and hit enter and we can go ahead and try this out with different
pictures.
So, for example, I could try -17.
And they should give us the -15 image from our images folder.
And you can keep trying this out for all the images we have in the folders corresponding to a certain
degree.
And with that, you have successfully completed this services project in which you implemented a Ross
service to send a number.
We wanted our fake robot to turn its camera to.
And since we don't have a real camera, we instead just returned a pre taken image based on the request
sent.
You also learned how to incorporate the popular OpenCV framework into your raw code to process and view
image data.
So I hope this gives you a little taste as to how you can implement computer vision in your robotics
applications.