And welcome back to the course in this section, we'll take a look at making a low light image enhancer.
Basically, our own vision of night mode effectively.
It's slightly different.
It works with just a single image and night mode, actually, and most phones takes a series of different
exposures of the image and then fuses them together.
Anyway, we're going to use a minute to do something that actually, I think is really cool.
So open Notebook 71 and we'll get started.
So I really explain what this model is going to be doing.
And if you want, I can actually scroll down all the way to the bottom to see the final result.
And you can see how well it works.
You can see this is the original image here can barely make out that whatever's on that image do on
that photo.
Now, if you use the pill, the pillow auto contrast feature, you can actually get this image, which
is better, but you can recover so much information and learn.
It almost looks like a regular daytime exposed photo similar to this bookshelf here and some of the
city's other items here, which we'll talk about later on.
So let's get started.
Sorry for all the scrolling, and I can make you a bit dizzy sometimes.
So we're going to be training on trading, on the L-O-L dataset, and that's not laugh-out-loud or League
of Legends.
It is the low light dataset that his example of what I just showed you.
It consists of pairs of images where it's normally exposed and then the dark and all new light, low
exposure image.
So it learns basically how to map those low exposure images to brighten damage.
So how do we do that?
Well, it's actually quite complicated, but for now, let's just import all libraries and download
our dataset and then create our TensorFlow dataset here.
Just some utility functions and we will be using to load and create datasets here.
Now we can talk about the that's model.
So the model actually is quite complicated, in my opinion.
It consists of a feature extractor that works across multiple skills, and then it has basically a mechanism
to exchange and fuse this information as it passes through the network, performing representational
learning.
And then it actually basically actually filters on different feature maps as well.
And it has a recursive design that progressively breaks down the input signal to simplify the learning
process.
So it's quite it's quite a good network, and you can see the architecture diagram here.
Now we're going to talk about the different features.
So this is the SKF module.
This is the module that performance fusing and selecting of different global features descriptors.
It basically looks at different resolutions here, multi resolutions or different skills and fuses them
together afterward here.
And you can read about it to learn if you want to learn more about how it works in detail.
And this is a function that creates the feature of fusion, that selective kernel feature fusion module.
That's what this block of code does next week come across the dual attention units or you, which is
used to extract features, as well as acting like a filter to allow more informative features to pass
further along the network.
So this part consists of a channel, attention and spatial attention.
And if you want to learn more, you can read this description of them there.
Next time we talk, we come across two blocks of code that perform those things, so we have a special
attention block in the channel attention block.
Another dual attention, a block which combines both of them together, as you can see here.
So now we come on to the multi scale residual block.
So this book is capable of generating especially precise output by maintaining high resolution representations
while still receiving the rich, contextual information from the lower resolutions.
So this may be more obvious or maybe consist of multiple fully convolutional streams connected in parallel.
So it allows information to be exchanged across these streams, which is quite cool.
OK, so it employs actually recursive residual design with Skype connections to ease the flow of the
information pooling process.
So this is that block of residual modules here in code, and you can take a look if you want to understand
it more deeply.
This here's the entire multi-skilled residual block there that combines everything you can see the new
SKF you to this here, too.
And now finally, we have the functions that.
Bring together everything.
So this is a recursive group here, and this is a burden that model that combines the previous models
that we created.
So if we get the final bernet model here, so now we can start printing our network.
So we're not using something so unusual a lost function called the year loss along with Adam Optimizer.
And for these types of networks, we're actually we're using peak signal to noise ratio of PSR as a
metric.
So that's how we measure how well this network is performing.
So we have those functions defined here.
This is a piece in our function in TensorFlow right here that we can use and we create a optimizes and
then just to model that flipped fit.
And then we plot the results.
So this model takes quite a while to train maybe a of a low of just over an hour.
But you can see the end of the peak nodes.
The peak signal to noise ratio is steadily going down.
That's really going getting better, I should say, in validation.
And in the training.
So that's quite good.
And we can take a look at our graphs and you can see the loss as well as apps and are increasing as
epochs increase.
So you can train for even longer and possibly get better results as well, although that seems sort
of oscillating around here.
So 50 might be the limit.
Maybe we can do other network enhancements to improve that.
So now we come across what we looked at in the beginning, which is some inferences here, so we can
run some inferences on some test images and test images.
There aren't a lot of test images in this dataset, so we just run to the field.
But you can see how well it works.
You can take this dark complex, and I think it's like all parkour, empty buildings in and then blowing
up here.
Nicole, could that look see actually extract all the colors accurately?
It looks like a low resolution daytime photo.
So does this cupboard with the pot still in this book can actually see the text and the in all the colors
and the stapler.
So you can see how well this is working and you can create this.
It's an EPA API if you want it quite easily with F+, and that's what we're going to do next.
Actually, sorry, actually going to do the capture of OCR tracker.
But then after what we're going to use the Flask Framework Web Framework in Python to create an API
as well as a web app.
So I'll see you in the next few chapters.
Also, the OCR tracker is actually a very cool chapter two, so stay tuned for that.
Thank you.