As I mentioned in the introduction to this section the theory behind Siamese networks is not all that
difficult.
It's more complicated than a CNN you would use for image classification but it still follows the same
steps.
We have a system of CNN as we set up a loss and then we minimize that loss.
Relatively straightforward.
The real challenge here is what do we do with the data.
Normally when we train a CNN for image classification we have a simple dataset of input target pairs
for x.
We have a single image and for y we have the class which the image belongs to.
But what happens with a Siamese network.
We don't have any label for a specific image necessarily or at least not the labels we will use directly
here's what will be given.
We're still going to get a list of images for each image.
We will be told their identity.
For example the first image belongs to subjects one.
The next image belongs to subject 2 and so forth.
But these are not the labels that the Siamese network cares about.
Remember that what we care about is if two pairs of images are a match or a non match.
So how do we do this
what we have to make our own dataset to keep things simple let's say we have three subjects and we have
two images for each subject.
Let's call these images a b c d e and f so A and B belong to subjects one C CND belonged to subject
to and E and F belong to subject 3.
Now this becomes a counting problem.
What are all the possible ways we can combine these images into pairs if you recall when we have N images.
We should end up with and choose two pairs.
So let's see what we get.
We get a AP AC A D E and F then we have b b b b and b f then we have c b c and S.F. then we have D E
and F and then we have just e f.
And of course we have the corresponding labels.
So a matches with B C matches with D and E matches with f all other samples are non matches.
You will notice a problem which is inherent in the way that this data is created.
It's that we will always have a much higher number of non matches than matches.
In other words we will always have imbalanced classes.
So if we tried to train the Siamese network with the usual batch gradient descent we are going to see
a lot more of the non match class than of the match class
to solve this problem.
We are going to do two things first because we always have and choose two combinations of images.
There will always be many more training samples than actual images.
We don't want to store all those pairs of images in memory because that would be redundant.
We would just be storing the same images over and over again.
Thus we want to create our own generator and loop over that inside our training function.
This will allow us to generate the data on the fly and not have to store it in some gigantic matrix.
Second by running our own generator.
This offers us some flexibility.
What we're going to do is for each batch we'll take say 32 images from the match class and then we will
sample 32 images from the non match class.
In other words for every epoch we'll be looking at all the matches but only some of the non matches
so far a simple example earlier where we had the images ABC the E and F here's what we would do.
Only a b c b and d f are matches all the other training examples are non matches.
Now let's say we do a gradient descent with a batch size of six then three training examples from the
batch would be the matches a b c any f the other three training examples would be chosen randomly from
the non match class.
So let's say we choose AC DC and b f.
Of course this means that on each epoch we will always see all of the matches but only some of the non
matches in the following lectures we will look at the actual code to see how the process we just discussed
is implemented.