In this lecture we are going to discuss the theory behind the facial recognition approach.
We are going to take in this section at this point.
I hope you have given some thought as to how you might approach facial recognition if you had to create
your own implementation.
I think this is a worthwhile exercise especially if you've never seen any facial recognition approaches
before.
This is kind of a one time opportunity because once you do learn about how facial recognition is currently
done it will taint your thinking.
So take this time to think about how you would approach facial recognition without any external influences
OK.
So now that you've had the chance to think about how you might approach facial recognition let me describe
to you what most might consider a good first crack at this problem.
Facial recognition seems like nothing but image classification.
Suppose for example our school has 100 students and given an image of a face.
Let's suppose we want our neural network to tell us which student that is so we might build a CNN with
101 classes where the extra class would be for images of faces which are not one of the 100 students.
Now this is not necessarily a bad approach but there are some issues.
For example what if one student leaves the school or another student joins the school.
Now we would need to build a new classifier or what if we need our neural network to recognize millions
of people having a neural network with millions of output classes is generally not a good idea.
In this section we are going to take an approach similar to binary classification.
Suppose we have some neural network where we can feed in to images the output of the neuron that work
is just a binary decision.
Are these two images of the same person or not.
The architecture we will use for this approach is called a Siamese network.
This is of course named after the famous Siamese twins.
The idea is this we are going to have to CNN with the exact same shared weights.
Hence they are twins.
Both of these CNN are not classifiers so they don't have any classification head and they don't end
in a soft Max.
Instead the final layer is just a linear regression with no activation which just outputs some feature
vector
here's what we like these CNN is to do if we input two images of the same person even if they are different
images we should get feature vectors that are very close to one another.
So the CNN would have to be trained to ignore things like lighting conditions different angles different
face sizes.
Maybe one day you are wearing glasses and one day you are not.
And so forth.
In other words the CNN has to learn the features that are actually distinctive to your face not other
differences in images
on the other hand if we input two images of different people the feature vectors should be far apart.
So even if the two images have the same lighting conditions are at the same angle have the same size
and so on the CNN should learn how to distinguish those two faces using the actual features of those
faces.
You can think of this in terms of word embedding.
We know that after training an algorithm like words of velcro glove we can map words into a vector space.
Similar words appear close to one another.
While this similar words appear far apart.
For example fish should be close to carp and laundry should be close to dryer whereas fish should be
far away from laundry in the same way the Siamese network outputs face embedding.
There is simply a mapping from an image of a face to a feature vector
at this point.
You may be wondering once we've converted both input faces into feature vectors.
What do we do with them.
I said earlier that we would do something like binary classification.
Well in fact there's nothing stopping us from doing exactly binary classification.
So let's say we take these two feature vectors and then we calculate the absolute difference between
them to give us a new vector.
Then we simply pass this new vector z through a logistic regression.
In this way we are doing exactly binary classification.
Now you might wonder why should we take the absolute difference.
Well this is because if we input image one into the first year of that work and image 2 into the second
neural network it should give us the same feature vector.
If we reversed the images so we input image 2 into the first known that work and we input image 1 into
the second year of that work.
The difference between these two images should be the same in both cases and thus the feature vector
must be symmetric
another option here is to build the feature vector using something called the chi square similarity
which uses the formula you see here.
Basically it's just a different way to calculate the features you should confirm to yourself that the
chi square similarity is also symmetric
but for our coding exercises I would actually like to show you a different approach.
This uses what is called the contrast of loss and it accomplishes exactly what our intuition wants us
to do.
Remember the idea is that images of the same person should have feature vectors which are close together
whereas images of different people should have features which are far apart.
So how can we push the feature vectors closer together or further apart.
The answer is the contrast of loss.
So how does the contrast of lost work.
Well it helps to look at the formula.
Basically it looks like what you would get if you combine the squared error with the binary cross entropy
which kind of makes sense because this is a binary problem but we are also working with continuous feature
vectors.
First let's talk about how it works and then we'll talk about why it works.
So let's discuss all the variables.
What does D that's the Euclidean distance between the two feature vectors or the two embed NGs.
You can also think of this as the L2 norm of ever vector 1 minus f of x 2.
Next what is y y is the target it's 1 if x 1 the next 2 are images of the same person and 0 otherwise
next what is m m is what we refer to as the margin.
This is the key ingredient of this loss.
We'll discuss this more when we talk about why this loss works finally.
Note that this loss is specified for a single training sample which consists of a pair of images x 1
and x 2.
So if we have multiple training samples then we would just take the mean of this
okay.
So now that you know what each of the variables in the loss means why does it actually work.
Let's look at the first term since it's a little simpler.
It's y times D squared.
Remember that y can only be one or zero it's one when the two images are of the same person.
Therefore when both images are of the same person only the first term matters and we want D squared
to be small.
We want to minimize these square if both images are of the same person.
The other scenario is when we have images of different people when a Y is 0.
Only the second term matters.
Notice how we are taking the max of m minus D and 0 basically it say if D is bigger than M then m minus
D will be negative.
So just take the loss to be zero.
Thus it saying if we have two images of different people we want D to be large but as long as it's greater
than or equal to M we don't really care how large exactly it is.
So we don't want to push D to infinity just bigger than M is good enough.
What is the result of this loss.
Well hopefully if we draw a histogram of the distances we get when we feed in images of matches and
non matches it will look like this.
You can see that we get two distributions.
The distribution for the matches should hopefully be very close to zero and the distribution for the
non matches should hopefully all be equal to or greater than M.
The idea is once we can see a plot like this we know how to classify whether or not two images are a
match.
We can choose a threshold for example with this plot.
That might be zero point seven five.
Then when we plug in two images into our Siamese that works and we get a distance less than zero point
seventy five we can predict that they are a match.
If it's bigger than zero point seven five we can predict that it's not a match.
As a side note you can see that we are going to set m to 1 which seems to work well enough for this
problem.
You are of course encouraged to try different hyper parameters on your own.