Hi, guys.
Welcome back to the course in this section, we'll take a look at using metric learning to do image
similarity searches, which is a very important field in the vision or anything, to be honest, because
finding similar images is something that's quite useful.
We think of search functions, think of things like signature matching, the fingerprint searches,
that type of stuff.
This is a very, very useful technique.
So let's get started.
So open Notebook 64, and we'll begin to listen and scroll up to the top.
I just want to point out this notebook again comes from the Keros tutorial site.
You official site and author is Matt Kelsey.
So credit goes to him and what we're doing.
We're going to do use similarity metric living on sofar 10 images.
So in a nutshell, what metric learning aims to do is train a model that can embed the input the image
into a higher dimensional space, such that similar inputs or similar images are defined by basically
being close to each other.
So you can imagine how useful this is.
So if you want to learn more, you can check out these links here.
So let's begin the lessons of firstly, we look at all libraries that we'll be using the next next load
this effort and dataset.
And no, we're just going to display 25 random images from the dataset, and you can see an upsurge
of dog and cat, blah blah blah for us as well.
So what metric learning is going to do is provide the training data not as explicit X-Y peers, but
instead uses multiple instances that originated in the way we want to express similarity.
So another example we're expressing the similarity by using the same class of image.
So that's how we're going to do it.
And so let's begin.
So we just run this block of code here.
Next, what we're going to do is create our anchor positive pairs because what's happening here is that
in training, we're trying to create to both the input to a batch that consists of the anchor and the
positive peer.
So the goal of the network in training is to basically move the anchor positive pairs closer together
and further away from other instances in the batch.
So you can understand how that's going to work.
So let's continue.
So you can see this is an anchor of positive pins here.
You can see the pins.
I believe they're showing like this here vertically.
You can see the similar class cut their dog, frog, horse boats and truck.
So now we're going to define a custom model with a custom training was started with a training step
that first embeds two anchors and positives and then uses a pairwise dog product as budgets for the
soft max layer.
So sometimes it's a bit of a mouthful, but you can take a look at this code here, so restore the training
step here.
We get the anchor embedding there, and then we just get the gradients and gradient gradient tape to
optimize this gradients after.
And then we update and return metrics, specifically the one for loss value here.
Next, this is our embedding model here.
So we created this embedding portal based on these inputs here.
And now we can train this model.
So we just compile that model that fits with our and compares us to inputs to epochs are trained for
and you can see the loss of the end.
This is gone down steadily, and if we train for longer, it most likely would have gone down even lower.
So now we can take a look and see how it performs so we can run a test.
So let's look at 10 meters per example here, and we can see this is a collage of similar images here.
So you can see this is the top ten here, so you can see the most similar images to this first class
here, which looks like a cat.
All these are the images here, which so much cats and dogs, but you can see they all looks somewhat
similar in this in the sense, similarly for the boats here as well, and frogmen can definitely see
it as well.
Quite similar.
So you can get a it depends on what you use as a as a similar metric.
We just use classes here and there was a lot of variety in the classes, so.
So how do you know if this is performing well?
Well, if you wanted to get a quantified view of how the performance of correctness of the matches were
done, we can take the 10 samples from the classes and then consider the 10 neighbors as a form of prediction
so that.
You can see if the nearest neighbor is of the same class effectively.
And we can create a confusion matrix out of its here, and you can see in the end the confusion matrix
supposed to have a high diagonal here with the sensors, you can see it's doing OK.
It's not great, but that's basically just after training to two epochs.
Not very much.
So hopefully you enjoyed that lesson or image similarity search using metric learning.
I think it's pretty valuable, and you can build lot upon this for your internal projects or whatever
you want to do.
I'll stop there now, and in the next lesson, we'll take a look at image captioning, which is a very
cool lesson.
We'll take a look at using Keros to implement an image captioning model, so I'll see you in the next
lesson.
Thank you for watching.