In this lesson we will be streaming from the drone camera onto the webcam.
This wouldn't be an actual streaming but we will be loading the image from the camera.
Now let's see how we will do this.
The drone camera will be filming a video.
The binary will be sent using UDP through the 1 1 1 1 port to the computer.
The data will be processed using f f MPEG.
Here we have the f f MPEG Web site.
For those of you who are using a Mac you've already installed f f MPEG when you install the open C V.
For those of you using windows we've installed the f f MPEG individually.
If you're not sure installing go to the command line and type in f f MPEG to see if it runs we will
be using the f f MPEG command to run the video through pipe 0 so the process will start to decode the
video so it can run on the computer after the process finishes the video will be sent through pipe 1
and the image will be showing in the webcam.
That's the overview of what we'll be creating.
Let's dive right into the script to make it work.
First let's start opening up the drone manager P Y.
The file will be located under models which is under drone apps.
Since we're at the top of the script we'll import the packages that we will be using later import O
S and said process.
Make sure you put it in alphabetical orders then import third party packages such as CV to a CV and
not be as MP.
Now let's go down a bit and declare the frame size globally frame X which will be the width is 960 for
the Tello camera frame Y which is the height will be 720 if we set the frame size too big.
The process time will be longer.
This is something we want to avoid when we start face recognition.
So let's set the size to third by dividing the numbers by three.
Also we need to set the frame area which will be frame x times frame y
next set frame size which will be frame size times 3 when the f f MPEG is in process.
It will receive not only the first frame but also many other data such as a r b value to process all
these data.
We will need about three times the frame area size later on.
We will need to use the frame position.
So let's set the frame centre X to frame x divided by two
also set frame centre y to frame y divided by two.
Next we need to set how we will be using the pipes set command f of MPEG to f string FFL pen then followed
by H W a CCL where we choose to process on hard or soft.
This will be to auto next.
H W ACL device will be open C L and output will be pipe 0 P I underscore F.M. t will be where it to
determine the colour which we will set to BGR 24 size will be set to frame X and frame y.
Lastly hyphen F will be where we send the raw video.
This will be set to pipe one I've created quite a long code so let's divide these into different rows
so you can see it better when you separate the row.
You need to use the parentheses.
Now we will go onto the drone manager in it before we run the process.
We need to set self proc to so process P open and run the command.
F F MPEG to split with the space and return as a list
a study in will be sub process pipe and a C.D. out will also be some process pipe
self proc a G in will be set to self proc SCADA and also self proc.
A study out will be set to self proc a study out the video part will be 1 1 1 1 1.
Next we need to set a thread.
Self received video thread set to threading thread target will be self received video currently received
video doesn't exist.
So we need to create it later args will be set to self stop event.
Self proc AC in self host IP and self video port once the args are set.
We need to set receive video thread start to start the process.
Next we need to create the receive video scroll all the way to the very bottom of the script.
Here will define receive video followed by self stop event by bean host IP and video port.
Now we need to set how the binary data from the drone video will be received.
First we start with a width stock at we use self sockets so it can be used in other methods.
But in this case we won't be using it in other methods.
So we will use the w statement with socket socket and set socket a f I any t to use the network and
we'll use UDP.
So we set socket socket diagram.
This is as SOC video next.
Let's set the socket video using set SOC O.P. t set socket aswell socket and socket as O reuse address
and set one by setting this code.
Once the socket video is released the address can be reused.
So if we reboot the system the address can be reused let's go on and set the time out time.
Here we'll set it to point five seconds.
Next we bind the SOC video host IP and video port this should be a tuple type so don't forget to set
it between the parentheses.
We also need to set the byte every two 2048.
There wouldn't be any problem if we set it to 10 24 but in the drawn SDK it was set to 2048 so the system
can receive the large video while not.
Will it be a loop that goes on until the stop event is set.
Next try size and address will be SOC video receive from.
Into and set data to get the byte airy
both the size and address will be set into each size and address.
If we won't be using this we can set it to underscore if you wish to log the data you can leave it as
it is.
If the process takes over point five seconds we want to receive a warning.
So let's set socket timeout.
As exception and leave a logger so we get a warning.
Action will be received video
E X will be X after logging we will wait for another point five seconds to retry.
Here we set.
Continue.
So the while loop goes on another exception will be when there is a socket error.
If there is an error occurring then we will break the loop before we break the loop.
We want to log the error with action receive video and E X as the X
we just created the part where we see the video sent from the drone through the bind socket.
The data will be sent into variable data.
Next we want to connect the data to the F F MPEG pipe in set.
Try then set pipe and to write the data with its size we set the data size to 2048 but that won't be
the size of the actual data.
It may be a thousand or even nine whatever size the received data is will be set to the data size.
Next we need to flush the data into the pipe.
0 here we also want to catch errors.
So let's set an exception and leave a logger so we know what happened in case of an error.
We'll have to break the loop.
We just finished the part where the drone video has reached the pipe.
Once we've got to this point let's test the script set a logger here.
I'll copy the log error code and change it to a log info and set to show the data.
Next we want to set the stop function.
Once we stop the process we also want to stop the F of epic process as well.
The video received will stop once the thread stops so that's stop here at the bottom of the stop function.
We can set o s kill to self proc PD to 9 if you are familiar with Linux you might find it easier to
understand.
I'll show you in the terminal.
We command P S with the system that is running you'll see the process information.
These numbers will be the PD you can command kill 9 followed by the certain PD to stop the process.
This code can kill the process as we just saw in the terminal.
For those of you who are using Windows would have to import signal then set OS kill self proc PD signal
control C event to kill the process.
In my case I'm using a Mac so I'll comment out on these codes once you finish all the codes.
Let's run the script and see if we receive the data.
For those of you who are using a Windows device once you connect your drone you need to turn off your
firewall to receive Pt. 1 1 1 1 1.
Remember to disable the firewall settings.
Now I've just started running the script.
Let's go to the controller page.
Now press takeoff.
Once you press the take off button you'll be seeing a lot of log data.
This means we've succeeded in receiving the data from the drone.
We'll be processing on these binary data in our next lesson.
Now let's press the land button to land the drone.
Make sure that the log has also stopped.
Now I don't want all these log data to take up the space so let's comment out on the logger info here
or you can also delete the row.
I'll see you in our next lesson.