Welcome to the section in which we shall be mastering image segmentation by building a virtual outfit
change plugin with the 51 app.
The tool we shall build will take in as input this image and then output this other image based on a
given prompt.
So as you could see here, we are able to modify the boot color based on just this prompt and an inpainting
model.
Nonetheless, for the inpainting model to produce this new output, where we have the boot color which
has been changed, it takes in the prompt, as we've said already, and a mask.
And in order to generate this kind of mask, where we have all the other parts apart from the boot,
which are set to a pixel value of zero, we are going to train our own SEG former model, which is going
to get this input and generate a mask like this one, from which we could extract specific classes,
like in this case, the boot.
And with that said, we'll start by looking at data preparation.
Then we'll go on to dive into model creation.
Then we'll train our model and after training this model, we will make use of this trained segmentation
model to input masks like this into our inpainting model, such that we are able to generate new images
which have been modified based on a specific prompt.
Apart from using this tool as a visual outfit changer, we are going to make use of it in data augmentation
by creating a very simple plugin with the 51 app.
The 51 plugin will build is quite simple and straightforward.
The first thing we'll do is select a given sample.
Once this sample is selected, we'll go on to click on Stable Diffusion Inpainting plugin and then put
in the different values like the specific class.
Like in this case we want to modify the shirt.
We'll also select number of augmentations.
We'll pass in the prompt.
We'll have the number of inference steps.
We'll have the guidance scale.
And then we would click on execute.
Once executed we'll wait for a little bit.
And then we get our generated samples like this one we have right here.
We go from an input where this man was putting on a military green shirt to one where we have this man
putting on a designer shirt.
And with that said, let's dive into data preparation.