-: Hello and welcome back to the course on deep learning.
Today we're finally at step number four, full connection.
So what is this step all about?
Well, in this step
we're adding a whole artificial neural network
to our convolutional neural network.
So, to all of the things that we've done so far,
which are convolution, pooling, and flattening,
now we're adding a whole new ANN on the back of that.
How intense is that?
That is just,
that is something, that is definitely something.
And, so here we've got the input layer,
we've got a fully connected layer and output layer,
and by the way,
the fully connected layer in the artificial neural networks
we used to call them hidden layers,
and here we're calling them fully connected layers,
because they are hidden layers, but at the same time,
they're a more specific type of hidden layers,
they're a fully connected layer.
In artificial neural networks
hidden layers don't have to be fully connected,
whereas in convolution neural networks,
we're gonna be using fully connected layers,
and that's why they're generally called
fully connected layers.
And so basically, that whole column or vector of outputs
that we have after the flattening,
we are passing it into the input layer,
and here we've got a very simplified example
just for illustration purposes.
And what the main purpose
of the artificial neural network is,
is to combine our features
into more attributes that predict the classes even better.
So we already, in our vector of outputs,
in the flattened result from what we've already done,
we have some features encoded in the numbers in that vector.
And they can already do probably a pretty good job
at predicting what's a class we're looking at,
whether it's a dog or a cat,
or whether it's a tumor or not a tumor, and so on.
But at the same time, we know that we have this structure
called artificial neural network,
which is designed, which has a purpose
of dealing with attributes and coming up,
or dealing with features and coming up with new attributes,
and combining attributes together
to even better predict things
that we're trying to predict.
And we know that from the previous part,
so why not leverage that?
And that's exactly what the plan here is,
so how about we pass on those values
into an artificial neural network,
and let it even further optimize everything
that we're doing.
And so that's what we're going to be doing,
but let's look at a more realistic example,
because this one is a bit too simple.
So here we've got a better-looking
artificial neural network,
where we have five attributes on the inputs,
then we have in the first hidden layer,
we have six neurons,
in the second, or in the second fully connected layer,
we have eight neurons, and then we have two outputs,
one for dog and one for cat.
And so an important thing to talk,
for us to talk about here is that,
why do we have two outputs?
We are kind of used to having only one output
in our artificial neural networks.
Well, one output is for kind of
when you're predicting a numerical value,
when you're running a regression type of problem.
But when you're doing classification,
you need an output per class.
Except for the, exception is when you have just two classes.
Like, we have two classes here, dog and cat,
and we could've just done one output,
and made it a binary output,
and said one is a dog and zero is a cat,
and that would've worked totally fine.
And actually, in fact,
you'll see had Lon do that in the practical tutorials,
and that's how they'll be structured.
But at the same time, if you have more than two categories,
for instance, dogs, cats, and birds,
then you have to have a neuron per every category,
and that's why we're going to practice with two categories
in this example, so that we know what to expect
if we ever have more than two categories.
And so what's going to be happening here?
So we've already done all the groundwork,
we've done the convolution,
we've done the pooling and the flattening,
and now the information's gonna go
through the artificial neural network.
So, let's have a look at how all that all happens.
There's the information going through from the very start,
from the moment when the image is processed,
then convolved, then pooled, flattened,
and then through the artificial neural network.
All four steps.
And then, a prediction is made.
And we'll see how this happens in a moment,
it will be very, very interesting.
But for now, let's just say a prediction is made,
and for instance, 80% that it's a dog,
but it turns out to be a cat.
And then an error is calculated,
a, well, what we used to call a cross function
in a artificial neural network,
and we use the means squared error there,
or in convolutional neural networks,
it's called a loss function,
and we use a cross entropy function for that.
And we'll talk about cross entropy and mean squared errors
in a separate tutorial, and how all that happens,
but for now, let's just say we have a loss type of function,
which tells us how well our network is performing,
and we're trying to optimize it,
or minimize that function to optimize our network.
So, the error is calculated,
and then it's back propagated through the network,
just like we had in the artificial neural networks.
Its back propagated, and some things are adjusted
in the network to help optimize the performance.
And the things that are adjusted are as usual,
the weights in the artificial neural network parts,
so the blue lines that you see here, the synopsis,
then also another thing that is adjusted is
the feature detectors.
So we know that we're looking for features,
but what if we're looking for the wrong features?
What if this didn't work out,
because the features are incorrect?
And so the feature detectors,
remember those little matrices that we had,
that the three by three matrices,
they are adjusted so that maybe next time it'll be better,
and let's see what happens type of thing.
And, but of course it's all done with a lot of science
in the background, and a lot of math, and it's all done
through a gradient descent with back propagation,
so it's all not just random perturbations,
it's actually very thought through how it's done.
But nevertheless, the feature detectors are adjusted,
the weights are adjusted,
and this whole process happens again.
And then again the areas back propagated,
and this keeps going on and on and on,
and that's how our network is optimized,
that's how our network trains on the data.
And the important thing here is that the data
goes through the whole network from the very start
to the very end.
Then the error is compared,
so the error is calculated, and then is back propagated.
So same story as with artificial neural networks,
just a a bit longer because of that whole,
the first three steps that we already had.
And now, let's have a look at the interesting part,
the really interesting part.
How do these two classes work, because,
or how do these two output neurons work?
Because before we've always kind of had one output neuron.
What happens when we have two?
How does this situation
of classification of images play out?
Well, let's start with the top neuron first.
We're gonna start with the dog.
How do we, the main purpose what we need to do first
is we need to understand what weights to assign
to all of these synopsis that connect to the dog
so that we know which of the previous neurons
are actually important for the dog.
And let's see how that is done.
So let's say hypothetically we've got these numbers,
in our previous layer, previous fully connected
or in the final fully connected layer.
And again, these numbers can be absolutely anything,
they don't have to be that,
they can be any numbers, but just for arguments sake,
we're going to agree that we are looking
specifically at numbers between zero and one,
so it's easier for us to argue these things and understand.
And one means that that neuron was very confident
that it found a certain feature.
And zero is going to mean
that that neuron didn't find a feature it's looking for.
So, because at the end of the day, these neurons,
I like, if anything else on this left side
is just looking at features at an image.
This is already very, very processed,
but still it's detecting a certain feature
or combination of features on the image, right?
Before we, in the convolved step,
we had kind of recognizable features,
in the pool step, they're less recognizable,
then they become even less recognizable,
in the flatten image, and then they get combined and so on.
But nevertheless, this we are talking about here
certain features that are present image,
or their combination.
So a one, which has been passed, and this is important
has been passed to both the dog and the cat at the same time
to both the output neurons.
So a one means that for us, for our argument,
it means that this neuron is firing up,
it's really rapidly detecting that feature
that you know might be an eyebrow.
It might be detecting this eyebrow for, again,
for simplicity's sake is detecting this eyebrow,
and it's communicating that to the dog neuron
to the cat neuron saying,
"I can see my eyebrow, I can see my eyebrow."
And then it's up to the dog and the cat neuron
to understand what that means for them, right?
And so in this case, which neurons are firing up?
These three neurons are firing up, the eyebrow,
and let's say the nose is saying,
"I can see, I can see a big nose,
and I can see floppy ears."
And it's saying that to the dog and to the cat.
And then what the dog,
and then what happens is we know that this is a dog.
So the dog neuron knows that the answer
it is actually a dog, because at the end,
we are comparing to the picture or to the label
on the picture, and it knows is a dog.
So basically, the dog neurons gonna say,
"Aha so I should be triggered in this case,
so these are my neurons."
They're telling this signal that they're sending
to both to me, to the dog and
to the cat is actually a indication for me that it is a dog.
And throughout these lots, and lots, and lots of iterations,
if this happens many times the dog will learn
that these neurons do indeed fire
up when the feature belongs to a dog.
On the other hand, the cat neuron will know
that it's not a cat,
and it will know that this feature is firing up,
And this neuron is telling me it can see floppy ears,
floppy ears, floppy ears, but
at the same time it's not a cat.
So basically to me, that's the signal
that I should ignore this neuron
like, and the more that happens,
the more the cat neuron is gonna ignore
this neuron about the floppy ears.
And so basically that, that's how
through lots and lots of iterations, if this happens often,
so this is just one example,
but if this happens often, maybe a one, maybe a 0.8, 0.9,
maybe sometimes it won't fire.
But overall, on average, this neuron is lighting
up very often when it is indeed a dog.
The dog neuron will start attributing
higher importance to this neuron.
And so there we go.
That's, that's how we're going to signify it.
We're going to say that these three neurons
through this iterative process
with many, with many, many, many, many samples and many
many epox remember, so sample is rowing your dataset,
and epoch is when you go through your whole dataset again
and again and again through lots and lots of iterations.
This dog neuron learned that this eyebrow neuron,
and this big nose neuron, and this floppy ear neuron,
they all seem to really contribute very well
to the classification of what it's looking for.
And which is a dog.
So, that's how it works.
And again, these ears, and nose, and eyebrows,
those are very very approximate
or like, very far fetched examples
because by this stage in this whole convolution,
convolution neural network,
it is completely unrecognizable what they're looking for.
But at the same time, it is something in the features
of dogs, or cats, or whatever you're classifying.
And then, so let's move on to the next one.
Now we're going to look at the cat neuron,
but these we're going to remember.
That these weights are,
you know, they have, we've sorted out the dog.
So the dog is kind of like, pretty much
ignoring all these other neurons, 1, 2, 3, 4, 5.
But it's really paying attention
to what these three neurons are saying.
Now, what is the cat listening to?
Well, whenever it is actually a cat, right?
The, this is, this is an example
of a situation when it's actually a cat.
So you'll see that this, these three neurons,
0.9, 0.9 and one, they're saying something
they're saying something to both the dog and the cat.
And this is again, important to remember.
So, this output signal goes both ways.
It's the same, right?
It's saying one to the dog, is saying one to the cat,
but then it's up to the dog and to the cat
to decide to whether to take
into account that signal and learn from it or not.
And both the dog and the cat can see that this is a photo,
I should have put a photo of a cat here.
But basically, imagine a photo of a cat.
Both the dog and the cat
can see that this is actually a cat.
So basically, the dog is like, "Oh, okay, so these whiskers,
and these pointy triangle ears, and this small size,"
I guess or I don't, oh, maybe the, these, this type
you know how cats have these things in their eyes.
Their eyes are like little, they're not circles
they're lines or something like that.
Like cat eyes, basically.
"These cat eyes, they're definitely not working for me.
They're not helping me out predict,
because every time these neurons light up
the prediction is not what I'm looking for."
On the other hand, the cat is like,
"Hmm, that's interesting.
Every time these, this one lights up, it's
or most of the time it lights up,
it matches my expectation.
It matches what I'm looking for.
Okay, I'm gonna listen to this guy more than this one.
This one, same thing.
Every time it lights up, or most of the times it lights up,
I happen to get a good, I happen to be rewarded
for my prediction because I get it right, it's a cat.
Okay? So, I'm gonna listen to him more.
You know, this one useless to me because he's not actually
you know, like he's, he's not even lighting up.
It's a cat, but it's, he's not lighting up.
So the opposite is happening.
And this one as well, it's a cat, but he is not lighting up
so I'm not gonna listen to him.
But this one, when he, what is this?
The eyes, the cat eyes light up, we can see,
I can see that it's a cat, it matches most of the time.
So, I'm gonna learn from that and I'm going to listen
to these three guys more often than not."
And so basically the cat is listening
to these three and it's ignoring the other five.
And that is how these final neurons learn which neurons
in the final fully connected layer to listen to.
So the output neurons learn which of the fully,
which of the final fully connected layer
neurons to listen to.
And that's how they understand,
basically that's how the features are propagated
through the network and conveyed to the output.
And so, even though these features
of course don't have that much meaning
to them, like floppy ears or whiskers,
at the same time they do have some distinctive,
they are a distinctive feature of that specific class.
And that's how the network is trained.
Because we also during,
remember during the back propagation process,
we also adjust the feature detectors.
So, if a feature is useless to the output,
it's going to, it is going to probably be disregarded.
Because this doesn't happen over one
or two, this just happens
through thousands and thousands of iterations.
So with time, a feature that is useless
to the network is going to be disregarded
and replacement feature is useful.
And so, at the end of the day,
in this final layer of neurons,
you are likely to have lots of features or combinations
of features from the image that are indeed representative
or descriptive of dogs and cats.
And so, then once your network is trained up,
then we, this is how it's applied.
So this is the next step,
like we've already trained up our network.
Well this happens,
Let's see what happens when the, this network is applied.
So let's say we pass
on an image of a dog, the values are propagated
through our network, we get certain values.
And so this time, the dog and the cat neurons don't know,
they don't have the image of the dog here,
they don't know that it's a dog or a cat.
They have no idea what it is.
But they have learned to listen
to what is being shown here, right?
They have learned to listen
to, dog neuron listens to these three neurons,
The cat neuro listens to these three.
And so the dog neuron looks at 1, 2, 3
and says, "Ah-ha, these are pretty high,
so my probability is gonna be high that it's a dog."
The cat neuron looks at these three and says,
"Okay, these, this one is pretty high,
but these are pretty low. Interesting.
So, my probability is gonna be 0.05."
And then, and that's where you get your prediction.
So then, your first choice
for this neural network is dog, second choice is cat.
And that's pretty much it.
So, the answer is dog.
And same thing happens when you pass an image of a cat
you get new values and you can see
that even though this one's high, these ones are low.
And for the cat, this one's high, this one's high
and this one's a bit low.
So the probability here might not be as great as previously,
but still you can see that it's a cat of 79%.
And so therefore, the neural network is gonna vote
that it's a cat.
And so basically,
or the neural network is gonna conclude that it's a cat.
Voting is a term that is used for these guys.
So, these neurons in the final fully connected layer,
they get to vote. And these are their votes.
And again, we are just, for arguments sake
putting values between zero and one here.
These could be any values, but they get to vote,
and then these weights are the importance of their votes.
So this is, these purple weights are how
the dog neuron views their votes.
How much importance is it assigns
to these neurons, and the to those votes.
And this is how much importance the cats neuron assigns
to these, to the votes of these neurons.
And so these neurons vote, the dog and the cats based
on their learned weights, they decide who to listen
to, and then they make their predictions,
and then the whole neural network concludes
that this is, in this case, a cat.
And then that's your conclusion.
And that's how you get images like this,
where you have a cheetah
and then you have a cheetah class with
you know, like a high, high probability.
So this is, you know
the probability that the network has predicted
and these are low.
But these still exist, because
they're still kind of like a small chance
the other neurons are also listening to their voters
and they're saying, "Oh, maybe it's actually a leopard."
And the bullet train, very, very probable here,
scissors, you know, this one won
but hand glass was very close second.
And then stethoscope, because you could see
like these guy, this, this neuron, the scissors neuron,
the output scissors neuron,
listened to its voters
and it had the predominant vote overall,
but then the hand glass had a good outcome as well.
So, there we go.
That's how the full connection works
and how this is all, this all plays out together.
I hope you enjoy today's tutorial.
We're gonna summarize all of this in the summary as well.
And I'll see you next time.
Until then, enjoy deep learning.