In this lecture we are going to continue looking at Siamese cat pie.
This lecture will focus on splitting the data into train and test sets.
Previously what we did was loading in all of our images into a big numb pie array and we also have another
array to tell us which subject those images contain.
To start I would like to know how many images of each subject we have to do this.
I'm going to use the counter objects and passing the labels.
Now when we print this out we can see that most of the subjects have 11 images except the first subjects
which has 12 images.
The reason why we want to do this is so that we can select an appropriate number of test images
next we're going to grab a set of unique labels.
Next we're going to get the number of subjects which is just the length of the set we just created
now.
Here is one key part of this.
We're going to make it so that for each subject three images will be used for the test set.
Importantly what you do not want to do is make your data set first where you have a bunch of pairs of
images and then do a train test split.
The reason you don't want to do that is because when you create your pairs of images the images will
be repeated multiple times.
So if you try to do a train test split after pairing then you will have the same images appearing in
both the train and test sets.
So don't make that mistake instead.
We are going to decide first which images will be used for train and test then we'll create our pairs
out of those train and test images.
So since we chose three as the number of images to take from each subject for the test set that means
and test will be three times and subjects then and train will just be an A minus and test.
Next we're going to initialize the arrays that will hold the train and test images.
So we create train images train labels test images and test labels using the N train and test.
We just defined importantly realize that we haven't yet created our pairs of images.
This is all just pre work that we need to do.
Next we're going to fill up the arrays we just initialize.
So currently they are all just zero.
And the reason we need to do it this way is because all the images are currently randomly ordered.
So we're going to loop through each image in the data set and for each subject we'll take the first
three images we encounter and assign that to the test set then any image thereafter we will assign it
to the train set so we start with a dictionary called count so far this will count how many images for
each subject we've encountered so far in our loop.
So the key to this dictionary is the subject i.e. and the value is the count.
We also keep track of train index and test index so we know where to place the image we encounter
inside the loop.
The first thing we do is increment the count for the current label.
So we index label and then we add one and then we use get here so that the default value is 0.
If there is nothing there already next we check whether the count for the subject is bigger than three
if it is.
That means we've stored all the test images already.
So that means we should store this image as a train image so we store that image variable in the train
images array at the current index train index then we store the label and train labels then we increment
train index
then in the else block we do the same thing except for the test images and test labels.