Hi and welcome back.
That, of course, in this section, we'll talk about auto encounters and how they can perform representational
learning.
So let's get started.
So firstly, what are often quitters?
Well, it's an unsupervised learning technique.
Like I said, that is used for representational learning.
But what exactly is that?
Well, firstly, do you recall what our CNN filters actually learned when you trained a CNN on an image
dataset?
Well, it lends things like high level patterns, as well as low level features such as edges and blobs.
So what if we could exploit what a CNN learns about a dataset so that it acts as a method of compression?
So think about something.
When you train CNN on an image dataset, it learns so many things about the dataset that there must
be some way where we can use those features we can use, like combinations of those features to generate
an output.
Well, that's exactly what auto input do they learn to compress data based on the correlations between
input features.
So some applications of this include things like denouncing images or even audio image in painting,
which is fixing areas of areas that obscured or missing an image, as well as information retrieval,
anomaly detection and obviously compression.
So, no, let's take a look at the auto encoder architecture.
Notice there's a bottleneck layer.
Now imagine the whole objective of an auto encoder is that we want to shrink the dimensionality that
represents data.
What that means is if there's like a number one or number two, you know that a lot of features and
that a lot of patterns in that number correlate with each other, meaning that a lot of local pixels
are grouped together, so they do correlate.
So what?
That's what the bottleneck architecture does.
It seeks to find those correlations.
That's the embedded learning that's been done in that layer so that we can know represented in a much
smaller vector dimensionality so that we can actually compress the data.
So often coders do it very well.
That data has the data that has correlated input features, which is images actually, in case you didn't
know.
So let's talk a bit about this bottleneck layer.
The bottleneck constrains the amount of information that is able to traverse the full network.
It's kind of obvious here.
So this enables the hidden bottleneck layers to learn a compressed representation of the input data.
So that's basically what all of this is quite simple in a high level way of looking at it.
Basically, all we're trying to do is learn the input data by using less information here to store that
information.
So that's why we use the correlated features.
So what is the ideal auto and could it look like?
Well, the ideal auto encoder is sensitive enough to accurately reconstruct the image, meaning that
it is sensitive enough in that it can take different inputs and use that to generate the correct output.
However, it's insensitive enough to inputs that the model doesn't over fit on the training data, so
it's a tricky balance.
So we'll take a look at this when we when we train our own auto and quitters in PyTorch and Keros afterward.
So let's take a look.
A deeper look at the auto encoded architecture.
A simple auto and could architecture is one where it bottlenecks or constrains the number of nodes in
the middle layer.
It's quite simple.
No noticeable input, an output much in dimensionality.
They have to be the same output because we're trying to compress like that.
Say this a 28 by 28 image here.
We're trying to compress it by a smaller representation, but then express it back as the full 28 by
28 image.
That's because we're reconstructing the input and the output.
So our laws function here penalizes Reconstruction ERA.
So this allows a model to learn the most important features needed to reconstruct the image.
Now, many of you who have studied data science and machine learning may think that ought to include
is quite simple in principle two principal components analysis.
And in that guess you are right, principal component analysis is trying to learn the features with
the highest variations so that we can represent the data at a much less, much more compressed state
that is technically similar to what auto and coders are trying to do.
This is just another deep learning way of doing that same thing.
So let's take a look at these CNN auto encoder architecture now.
Given that our inputs in computer vision are images, using a CNN takes makes total sense because the
Cavaliers provide much better performance because they're able to compress that image structure and
linear correlations within that image.
So that's why sorry about that.
That's why Convolutional is ideal for constructing a auto encoder for images.
So let's take a look at how we train not to include a.
So the training process is quite simple.
However, there are a few differences, so let's take a note of those differences here.
So the target data is the same as the training data.
Look at this here.
This is the sheriff's auto include a structure extreme.
Usually we have extreme and x test.
Here we have extreme and extreme.
That's why the target is the same as the training data.
So what does that mean for the rest of it?
Well, it means also for validation.
X2's is also the same as X2's.
And why is that?
Well, that's because we're trying to test how well our encoder decoder model works.
So when I say and could a decoder model, what we're doing with an auto encoder is that we're taking
an input dataset, shrinking the dimensionality or the representational size of the dataset and basically
storing the information of that data.
So we're encoding, let's say, the number seven in a smaller vector.
No.
And then we can use the same encoder decoder model to now rebuild the input from that smaller representation.
That's what an auto input is doing.
So the lost function we can use for this can be either something like binary cross entropy or even means
quit error.
So let's take a look at the limitations now of what to include is auto included as a lossy meaning that
decompressed outputs are degraded bits in original, mainly because we have less information to store
that data.
That's basically what lossy means.
It's data specific, meaning that it looms a representation only of a specific domain so that, no,
we can't apply it or to include it, that we have trained on c 100 handwritten digits to the two letters.
It's not going to work.
So it's it's domain is data domain specific.
So next, we'll take a look at building an auto and could is input keras and PyTorch, so I'll see you
in the next lesson.