Hi and welcome to the lecture on Sony's networks, where we talk about how Siamese networks can be used
for tasks such as image similarity.
So let's get started.
So if you're unfamiliar with the term Siamese, Siamese refers to conjoined twins that is twins that
were unfortunately joined together, but sometimes even sharing organs so similarly to wood, Siamese
and Siamese networks refers to networks that are connected.
They consist of two or more identical some networks, and these same networks all share the same parameters
and widths the parameter.
But it's a myriad of both networks.
And which means that when one network updates to which the other one consequently updates its way,
it's exactly the same value as well.
And that's an important fact.
So now let's move on to what are Siamese networks used for?
Well, they are very, very useful in comparing images or gathering image similarity.
Things like signature verification, face recognition and fingerprint matching are all examples where
we can use some these networks, and you would be very familiar with face recognition, which has been
a bit controversial of late.
So Siamese networks have a simple binary output.
Yes, if the images match or new, if they don't match, and here's an example of some signatures that
do much and some signatures that don't match.
So here's a high level diagram of what a Siamese network's architecture looks like.
Firstly, we have the image being fed here.
We have two images one the images that we're trying to compare.
So we have an image one image two.
And then we have the confirm that were created.
And these are identical convolution networks.
You remember that.
So what is convolution that looks produce is an encoding that's towards a representation of this image,
so it learns one that works learn to to create and encoding that we use later on now to compare in the
different singlet.
Now the different thing is where we actually compared the values using something like Euclidean or cosine
distance, and we use a lost function, typically triplet loss or contrastive loss or even binary cross
entropy loss.
And this produces a similarity score.
So if the similarity scores low images are similar, meaning that they're very, very close match.
Otherwise, if the similarity score is large, kind of reverse, kind of unintuitive, but just remember
that so the similarity score metric looks.
In this case, the images are different.
So a low score means the images are similar.
So here's another diagram of a typical Siamese network.
You can see the convolutional layers here.
Then it has a fully connected layers.
Then it goes to the contrast of loss here.
This fully connected layers have produced the embedding that that is towards the image and that converts
this image to this embedding here.
Likewise, it takes another image to image that we are comparing here and here convergence in exactly
the same but exact same architecture converted into another embedding here.
And then we use the contrasts of lost function to compare using Euclidean or cosine distance, and this
outputs a similarity score at the end.
So here are some key points to remember when thinking about a Siamese network architecture.
Firstly, we noticed the output of the convolutional layers in a Siamese network, a flat matrix.
This is an illustration of what I'm talking about right here.
Remember, we have to some networks put convolutional convolutional networks if we're using it for images.
And this concept works produced a one dimensional vector.
That's basically the embedding layer or the including, which can be considered the features that have
been extracted from the image since we're using a conflict to produce that encoding.
Now what do we do with that encoding?
Well, we feed of bulletin cuttings.
Remember each image produces and encoding.
We feed it, you know, into different things.
And that's where we can use something like Euclidean or cosine difference to find the difference between
the matrices produced in each network.
We can then use a loss of function, though, to assess whether the same means that both has made the
right decision from the comparison.
So let's talk about the lost function.
The tree common loss functions we typically use in saying these networks.
Firstly, there's a trip that lost function.
This is where a baseline input here showing as the anchor in this diagram is compared to a positive
input.
That's this one here.
And the negative input, that's one that's different.
And what we try to do is try to maximize the distance we can see.
We try to minimize the distance or explicitly between the positive being the anchor and then maximize
the distance between the negative peer and.
Yanco, right, dear.
Secondly, there's contrastive lost, no, the goal of a Siamese network is to differentiate between
pairs of images.
So contrastive refers to the fact that these losses are computed, contrasting to a more data point
representations.
And you can see an illustration of the contrast of lost concept here.
And then lastly, we have binary cross entropy.
So given that the output of the same is that which is binary and that is similar or dissimilar is kind
of intuitive that we can use something like binary cross entropy lost function, as it might be an obvious
choice for these types of networks.
So hopefully you have an intuition and understanding of Siamese networks.
This is a high level overview.
Next, we're going to discuss how we can actually train the same these networks.
So stay tuned for that.
I'll see you there by.