Hi and welcome back to the course in this lesson, we'll take a look at building our very own super
resolution again or her again in Paris.
So let's get started.
So open notebook territory like I just did and we'll begin to listen.
So firstly, the tutorial, inspiration and credit is attributed to this author here, and this is the
link for the actual material on the official Keros example site.
They have so many good tutorials there, however, there are a lot of them are a bit hard for beginners
to understand.
So what we'll do in this course is that will go through maybe a handful of these tutorials because these
are research paper implementations of some pretty state of the art networks.
So let's get started with the lesson.
So, but sorry again, we're going to recreate it here is called the.
It's called the efficient subpixels CNN, and it was proposed by XI in 2016.
This is the original paper here by the water domain.
Also here.
It came out in 2016, and you can browse this people, if you want to get a good, deeper understanding
of how this summer again works.
So let's continue with the lesson.
So we're going to try and implement this model on a small dataset called the BSD 500.
That's from Berkeley's Computer Vision Group.
And you can take a look at some of the dataset information here.
And now we'll go back to the lesson.
So firstly, let's set up and load our libraries and data set, but we're not at the low end of the
dataset here just yet.
We're getting the dataset over here.
So let's download our data said while these libraries are loaded.
This will take about roughly 20 seconds, so just be patient and we'll have our data downloaded shortly.
OK, great.
It's complete now to roughly 24 seconds.
So now what we do, we have to split that data set into a training and validation separate sets.
So we'll do that here, and this will have some parameters be defined here.
So we're setting the crop size two to three hundred setting an upscale factor as well as an input size,
which is a crop size over the upscale factor, as well as a batch size.
So we'll use image data set from directory here to get our data set, and we just set the validation
split two point two and then we go, So let's create that.
And that's done, and you can actually see the summary below of what we get from using image data set
from directory.
So we have 500 files, all belong to one class using 400 for training and then using 100 for validation.
So now will this scale create a function to reskill data between zero and one?
So then we just apply it to all data set that we've created here the validation dataset and the training
dataset, so we can scale our dataset accordingly.
So then now let's visualize some batches from that data.
So we can see we picked some batches here is a weird looking animal, looks like some sort of monkey,
but I doubt that's a monkey with a lizard.
But there's a dog with some people on snow.
So you can see there's a very wide variety of images in this dataset, which is good.
That means our model will generalize well to new images next.
All we do here is just we can get to test image parts here using these functions.
So we get all of the image names, well, all the image parts for the test images right there.
So let's run that.
Next, we'll convert our images to the volume color space, which is a more human way of perceiving
images.
And it also has some advantages over RGV.
So we'll be using that for this shooting network, and you can train on any sort of color space.
You can create an RGV.
Typically, we always do that.
However, you can train on HSV or white movie as well.
Next, what we'll do will create some helper functions here to process the input that that means it's
converted into white image here.
And then we also have process to target as well, which does something similar, but only returns to
white channel because that's the only time we're looking at here.
And then we use a TensorFlow pre fetch function to create some data loaders that allow us to load data
quite quickly during the training process.
Next, what we'll do We'll take a look at some of our training and target data, so you can see these
are first images.
Here are low resolution images that we want to upskill to these.
These are the ground truth images here.
So the network is going to learn that with CNN.
So typically when I said this is a gun.
Guns, guns have been used for super resolution.
However, this specific gun isn't really based on generative adversarial networks.
It's based on convolution networks.
So it's more for our CNN.
So, but nevertheless, I put it in the gun section just because it's commonly associated with guns.
So now we can move on to building new models so you can run this.
I've already run all this courthouse, so you don't I don't need to do it again.
I don't want to mess things up too much.
So I just go through the code here.
But remember to run every cell block as you go along with this lesson and you can see to CNN, it's
just kind of it's quite simple.
Comes on.
Come on on top of cons and then you can see we have different arguments here where we have really activation
orthogonal panel of Initialized, which is a different type of initialization, as well as we use padding
for this same padding, so we don't lose the feature map size as we progress to the network.
Next, we define some more utility functions.
This one +2 results, and this one gets too low res image here.
This one upscales the image, and now sorry, we can create some callbacks to monitor trend filling.
So we just create a callback class here, and we just stored in our value.
That's peak signal to noise ratio, which is a metric we use to evaluate super resolution type networks
for the performance.
And then we just have one epoch end on test batch so we can print and get different training accuracy
or training performance metrics during training process.
Then we can just finally create the model here and get the summary and then define everything here with
the optimizer and lowest functions, as well as getting set of setting a little callbacks and the checkpoints
so we can see a model.
SUMMARY It's quite tiny.
It's fifty nine thousand trainable parameters, which is very small because it's only conflits.
That's why we actually trained for filters, not for accuracy in these types of networks.
So now we're ready to train the model.
So this model takes a little bit of a while to train.
So this is a training process, the results of my previous training experiment.
And you can see this is a prediction here.
And this plotting function that we defined above allows you to zoom into an area and expand it here.
So that's quite cool, and you can see this as a lost function, as the progress of the smaller and
smaller.
And you can see the mean percent are for each layer, and this should get larger and actually twice.
And this should get larger and larger as we progressed, should a network doesn't change that much,
it seems after a while, but that's OK.
And let's look at this.
These are all the results of quite a bit.
Now we can look at displaying the model prediction as well as plotting the results here.
So you can see the piercing hour of low resolution images and high resolution image is twenty nine point
eight five percent of the predicted high resolution.
That's 230.
So that's interesting.
So you can see now this is a low resolution image here versus the high resolution images versus the
prediction.
So let's take a look at all three of them together.
You can see in the high resolution, some would say it's a bit of a defined logo with the individual
looks that grips on this model.
It's all a blur here.
It's still a bit of a blur.
However, it's slightly sharper, like this line has slightly sharper than this line.
So it's a bit cleaned up.
A bit more defined, but not that great.
Then we can look at the images here.
This one is a woman holding a wine glass, and you can see in the prediction vision everything looks
slightly sharper, but you can see it on your video stream, but it's a little bit sharp on my end.
But you do have the source notebook for this, so you can analyze it when results.
So that's it for this lesson on super resolution that works, this one doesn't make a huge difference.
I may record another video lesson with a better, more advanced super resolution again.
So for no, this is it for this lesson.
And if you want me to record that video, just let me know I'll put the notebook up in the course.
Resources.
So you guys can have access to that as well.
So thank you for watching.
And that's it for our super resolution guy.
And listen, thank you.