Hi and welcome back.
So in this lesson, we take a look at a problem that has basically stumped computer vision practitioners
and engineers for four years.
For decades, maybe.
And that's how to get dumped from an image from an monocular rg b image, meaning that there's no adept
information like a LEIDER or RG b d comer of the distance for depth can give us that depth, like a
time of flight sensor can give us a depth of the image of things in the image that makes individual
pixels.
Even however, most cameras pretty much all camera phones don't use that, and when you're including
the image and video cameras as well CCTV.
So we just deal with plain RGV images and we need to find a way to infer depth from that image depth,
meaning the distance.
So let's open that book 63, and we'll take a look at this less than here.
So credit for this notebook goes to Victor Basu.
This is another official Keros tutorial online, and you can see this is what we're trying to do with
trying to get depth estimation so you can see this is a regular RGV image here.
Apologies for the low quality.
It's just a small image that's blown up now.
This is a ground truth depth prediction here, so you can see this wall is closer to you and things
we read.
And that's what the ground truth labels look like.
And this is what a prediction looks like.
And it's actually quite good.
Very close.
It's just a bit fuzzier around the edges sometimes concede four different images.
Here's another indoor image.
This dataset is composed of, I believe, of indoor and outdoor images.
So let's begin.
So to run the setup, just lower your standard libraries that we're going to use, we download the data.
This takes just over a minute, and it's basically a subset of the diode, which accounts for dents
in the one outdoor depth dataset.
The original dataset is eighty one gigs.
So this we just training on the validation dataset, so it's much smaller.
Obviously, our results aren't going to be as good, though, but for the interests of time and tutorial
purposes, we will use that dataset.
So we prepare the dataset here.
We just point to the right parts and images.
So we have to get the image, the depth and the depth to mask and visualize those things shortly.
Next, we just prepare the hyper parameters as well how much epochs and about size and lending rates,
height and widths.
Next, we have our class data generator where we just basically it's like a training section.
I guess you can consider it here or pre-processing the depth map.
All of those things.
Actually, this isn't the training.
Sorry, but this is just some of the functions that we will be using.
OK, so now we can visualize some of our samples here.
So let's take a look at that.
So you can see this is the original image here.
This is the ground truth labels.
You can see the corners of first part of the image, and as you go closer here, these are closer to
us of similarly.
Here you can see the she and foreground exclusive ones in the back.
People didn't talk or read.
This wall is further away from everything else and goes gets flipped as we move to the right of the
image and so on and so on.
You can see this one.
This actually has a lot of variation in it, even though it's not that much depth difference, but it's
needed to illustrate and teach our model how to infer depth from these images.
Actually, that I thought this was a mirror just now is actually a window to in the room.
So that's why everything is much darker red and you can see in the a corner point as well.
So now we can actually visualize the depth because remember, it's in depth image so you can actually
see what it looks like here.
So this is one of them.
You can actually see the depths out of it.
So it's pretty cool, isn't it?
Pretty images?
Next, we're going to build a model, and the basic model is a unit model.
So that's this is where we define or downscale block here and the upscaled block to remember how the
unit has a bottleneck in the middle.
And that's what we have here.
So we did find that the next we defined the loss functions here.
So we have three different losses.
We have structural similarity index ss i am l one loss or point was dumped in our case and depth smoothness
loss.
So we define of loss functions and calculate them there and next.
We're ready to train tomorrow model, so we just set about all our training and data generators, and
then we run a model that fit to train the model for two epochs and the train trains quite quick on the
GPU here and you can see voltage and low start up 1.4 and then goes on to one roughly and then goes
on all the way to point one eight one one nine.
That's not bad.
Actually, it's better than the training wheels, to be honest.
So now we can visualize the output, so we take a look with loads of images here, and you can see this
is the real image.
Head Hajime.
The image that we're training on this is the mess that is supposed to generate using a unit model.
And this is adept.
So you can see it's not.
It's not that bad.
It's OK.
This one is, I think this one is actually reversed.
If.
Yeah, this one was actually visiting the foreground.
Is this and this is the background, unfortunately.
This one is a little bit better, actually, although it does think something on the corner his for
some reason this one.
The ground should looks quite weird in the labeling, but I guess it's accurate.
Probably doesn't need to be such granular.
Some big changes here.
This one is actually probably better from analysis point of view.
I like it because I put a ring here.
This isn't for the back, although I hope it's maybe just a red from this being looking darker red over
it.
This one, this one is okay, but it has different colors for the same object.
It shouldn't really be and so on.
So you can examine all of these here.
So there's a lot of possible improvements.
However, remember we didn't train this for very long, and we didn't treat this for the full dataset.
So there's a lot more improvements that the researchers didn't do to get the best results.
These are some of the references you can take a look at to see to learn more about this topic.
It's a very interesting topic.
It's a very hot topic as well, although there are some very good networks now that do depth estimation
quite well.
So once you have a reference point, you can basically get the depth of any object to a fairly, fairly
rough, but good enough accuracy.
So that's it.
I'll stop there and I'll see you in the next lesson.
Thank you for watching.