All right.
Let's learn how to segment anything for inpainting.
So that's the original image.
And we're going to be able to get an exact section of the image just by telling in the prompt what we
want to get in this case Rhino's head.
So it's going to create the mask for us.
And then we can pass that in to generate a new image using that specific space.
This is a little bit messed up.
This one it's got an extra hand.
But I'll talk you through some of the reasons why that might be.
All right.
So that's what we're going to try and accomplish.
And let's get started.
All right.
So let's run pip install file client.
And we also need to get the environment variables.
So you need your API key in your env file.
And then we just need to make the call to file.
So file has the segment anything model which is open source model by Facebook.
And we're going to be using that.
First we need to get the file client and requests.
And then we're going to create a function just to give us updates when we.
When we're waiting for the response.
Is instance update.
Client in progress.
And then I'm just going to say log in to logs print the log message.
Okay.
So that's just going to give us updates as we go along.
We also need to upload the image itself.
So I'm just going to say image URL.
We can we've got like a Rhino image that we're going to upload.
And that's just to get the URL from, from the image that we can pass into the next model.
So the next one we're going to do is actually creating the segment anything result.
So just going to say result equals false file client subscribe.
The model itself is called Sam.
That's one that I found that works really well.
And the prompt here is just the rhino's head.
I'm also going to add another couple of features.
So use grounding Dino or Dino.
That is just deciding whether you're using Sam or grounding Dino to do the object detection.
I found that this worked a little bit better for me, but you can experiment yourself.
I've also added fill holes and expand mask.
That's because I want like a full kind of blob rather than it defining the outline too much.
Then I wanted to make a bigger space, essentially.
And if you do that, then you get the result.
You can also get the URL and let's just say let's print out the original image and the mask.
We just kind of see how that looks.
Let's see if this works.
Okay.
That's the original image.
And there's the mask.
So you can see it's drawn an outline around the rhino's head.
And then drew the outline just like magic, which is really cool.
Now why do you need this?
If you want to do inpainting, you need to be able to say exactly what part of the image you want to
make changes to.
Here we can do that.
We can say a line in the suit, and then we're calling the inpainting model with the image URL and the
mask URL.
So this is like a programmatic way to get out the image that we.
The section of the image that we want to change.
And that's super helpful.
And we're just going to print out here the transformed image.
So I'll just show you what that looks like.
And it's going to take.
And it's only going to generate within that white space right.
Like it doesn't generate it doesn't change anything else about the image.
And you can see here like sometimes it struggles to fit something in.
So you can see it's like a bit of a gap here.
That is one downside of inpainting because that's just the way it is.
But it depends on what your use case is.
For example, if this is a little bit weird because it's a rhino going to a lion to have different shapes,
but if it was like a human face, it would work a lot better.