In this section, we've just seen how to create or how to edit existing data and create new data from
that, um, already existing data.
Now this data may be data we the model has seen already, or data the model has not yet seen.
We're going to move to the next part where we are going to look at data augmentation.
And we will end this section by creating a plugin with 51.
That's essentially a 51 plugin.
So the idea of data augmentation here is we have supposing an input image like this one.
Then we edit this image.
In this case you see how the boots are edited.
You can see this.
And then in this other example you could find that the code is edited.
So what we're saying here is we take um an input pair like this one which is essentially composed of
the input image and its corresponding mask.
And then we edit this image so that we have this new image.
We also can create this other new image.
And then we repeat the same mask.
So we take this mask and then we simply repeat it.
So now you see that we're going from one sample to to two other or to three samples.
So we we triple the sample number by simply editing this image.
Now it should be noted that after editing this specific image right here, we could go ahead and um,
carry out the editing on this one so we could end up with an image where we have the boot turned into
red, the code turned into green.
Uh, maybe the sunglasses changed into some, let's say, red sunglasses, maybe the person's skin,
um, turned into black and so on and so forth.
So we must not always edit from this original image that said, this way of creating new samples or
augmenting our data via editing like this permits us to create much better and diverse, um, augmentations,
as now we could even change the background so we could decide and change this whole background.
We change, um, each and every region here, and we have a whole new image.
We could even change the the person's face and say, for example, um, uh, we want a man.
So now you'll be getting, um, a different background.
You get, um, uh, a man like suit.
Although a man could put on this kind of coat, um, you could change the hairstyle and so on and so
forth.
But nonetheless, it should be noted that, um, the shortcoming of this method, especially with a
technique like um, or especially when dealing with segmentation, is that the newly edited image,
uh, may not match with the the mask.
So we may end up with a slightly noisy data set.
That said, let's dive into the code and see how we could, um, create our new data samples from,
um, our original data set.
We can now get back to the code, and you should note that the the code for the data augmentation is
going to be very similar to that of the image editing as here.
The only difference is, unlike with the image editing where we have the model like um, here we had
the model which predicted the mask.
That's our trained, um, former model which predicted the mask.
We are going to make use of the already existing mask for a given, um, data point or for a given input
image.
So if we have an input image like this one, we'll make use of its corresponding mask, um, which is
given in the data set to generate this, uh, mask, which will then be added with a prompt, um, passed
into the inpainting model to create a new data point.
So again here we have the same model ID.
We have this create pipeline, which essentially is the same, uh, we've seen already.
Then we have this generate inputs, which is different from what we just saw on the image editing.
And you will notice that the difference here, um, lies the fact that we take off this part, which
includes the model.
So all this resizing, we don't really need all this.
That's you see here we, um, resize, we normalize, we pass into the model, and then we resize again,
um, before creating our mask.
But since our data or our data set already comes with this, we take off all this.
So getting back to data augmentation, um, scrolling down you see here we have the image path and the
mask path which is passed in together obviously with a with a mask ID.
So, um, in here we have the image path.
You see, you read that image path, you have the source image.
You have the mask, which is also read just as the the image path was read.
This read from our data set, um, then obviously specified as grayscale, then the same processing
we did previously to select a particular, um, category is what we, we, we repeat here.
So no change in the code.
Then we convert this to Pil images.
So that's it.
Uh, once that's done, the next step will be again similar to what we saw with image editing.
So here under image editing, we simply, uh, make use of our pipeline.
And then we pass in the prompt, we pass in the image the mask.
And then we got our output image.
So it's the same thing we're going to do here.
Let's scroll down.
Here we have the pipe, um, our pipeline.
We pass the prompt.
But now we have this two parameters which we've added the guidance skill and uh, number of inference
steps and the generator.
Now, it should be noted that we could also do the same.
We could also add this here in our image editing.
So we could we could add the the guidance guidance skill.
And we could also add the number of inference steps.
So this three extra parameters have been added.
Now this guidance skill shows the model how much it should adhere to the prompt which we've passed in
here.
And also obviously um increases or as we increase this guidance skill, um, increases the image quality.
Then for the number of inference steps, if you want to hire or increase the number of inference steps,
increase the image quality while reducing um reduces the image quality.
Nonetheless, stable diffusion works well for with even low number of inference steps.
Then for the generator this is for reproducibility.
So if we set your random seed, or if you set your seed to ten, then each and every time you generate
the exact same image.
So that's it.
Um, um, we have the usual image and the mask.
Then one thing we want to do is we want to add this, um, number of images per prompt, meaning that
for every time we call this arc paint method, we could say, for example, we generate ten images and
that's it.
So we put all this in a list and then we return the value here.
We're just we're just taking in the Or we're just taking out from this just a single one.
So we could take this off and take uh, retrieve all the, uh, edited images.
Um, the next we have this hash method which essentially produces, um, a value like this one.
And the reason why we needed this method is simply because here when we create, when we create our,
our extra data, let's say we have this extra data, let's have this here.
Let's say this is called 00033.
Let's increase the size to say 100 okay.
So let's say this is called 003.
Then in that case if we want to create some extra samples or if want to augment our data, we could
call this 003.
Now underscore the hash value we had from here so we could copy this.
Go back here and paste it out.
There we go.
So this will be the file name for this new um data.
So we have this copied and there we go.
Let's just change this a little so we could say this is E.
There we go.
So now we have this three samples with three different um, file names.
So that's essentially why we need this hash.
Um that's fine.
The we'll just keep this part and then go straight into this transform sample method.
What the transform samples method does is it takes in a sample from our 51 data set.
It takes in um, the category ID or the specific category in this case quote.
It takes in a prompt and then it modifies that sample based on the prompt.
And the corresponding category.
Then creates a new sample which is then added to the original 51 data set.
So looking at this um, method, obviously we have the the hash which we create.
And then we have our pipeline which we get using the create pipeline method and the model ID, um,
and passing the model ID into the method.
And then now we generate our inputs which we'll get into our inpainting model.
So the way we generate our inputs is simple.
Since we get a sample from our 51 data set.
Remember our 51 data set uh, which we've seen already so we could get back here.
You see, this is about 51 data set.
Um, let's scroll down and then, um, get back to the code.
So as we're saying, we have let's.
Let's just do data set data set head.
Data set head.
So we could, um, get some samples.
There we go.
You see, we have this sample.
And then from here we could get the file path.
So as we're saying here we have this file path we should pass into our generate inputs method which
will permit us generate an image and a mask.
Um not forgetting the, the ground truth or rather the mask path.
So again here you see we have our given sample.
We have the ground truth mask path.
That's it.
So we specify this tool and then we obtain the input.
So if we get back here uh we had seen that to generate this we needed this image path.
We need a mask path.
And then this mask ID.
Now this ID will depend on, um, the specific category we want to modify when carrying out the augmentation
or the editing.
So here we have a the select class.
Select class.
In this case if it's code then label to ID will convert that into um the specific id.
So if it's code then we should get something like 13.
So that's it.
And then now we pass this inputs into our augment method.
So here we have our pipeline.
We have the prompt.
We have the image.
We have the mask.
We have the gradient scale.
And we have the number of inference steps which we pass into our paint method which we've seen already.
And now that generates the images.
So we now have our, um, new newly generated images.
The next step will be to add this into our data set.
So we simply write this into our data set I have the samples file path.
So if we have a file path like this one right here, or rather this one, let's say we have a file path
like this one.
Let's copy that.
So we could better illustrate, um, or better show you what exactly is going on here.
We if we have this file path, then we take off one, two, three, four.
Then we add the underscore, we add the hash and then we have dot png.
There we go.
And then we write this or this output which is from our pandas.
The the image which we get from our augment method as an edited augmented image is what is reading in
this file path.
So now, uh, we will not only have our image 0018, you also have image 0018 hash png.
Now because we do this for the images, we must also do the same for the PNG masks.
But for the PNG mask is just a copy.
So we'll just copy um what we have in this mask path.
So we'll copy this, this image here or this mask, um, into the same um, folder, but with the difference
that we're going to change its name, its, um, file name.
So we add this hash here, the same hash we used here.
So let's take this off.
That's it.
And then now we could display our output at um.
And it doesn't suffice only to, to say copy this new data into our data set.
Uh, we also need to create a new, um, 51 sample.
So to create this new sample here we have the file path for the image.
Obviously it's going to be um image 0018 um underscore the hash dot png.
So that's it, the same one we have here.
And then for the ground truth, uh, we specify that it's um segmentation.
So we expect to, to, to have a mask or a mask path.
And so here we have our mask path which is the same path we had specified here.
So now um, having this, we are telling 51 that we have this new sample.
And then here we're going to simply just add this new sample into our original data set.
So let's run this and then see what we get after running here.
The output we obtain.
You could see we have this green coat.
Um that's fine.
Let's go ahead and check um our data set.
Let's increase this so we could see this better.
So you see we have this new, uh, mask added.
And also we have this new image added.
Click that and then you could check out the mask.
So now at this point we have our image and its corresponding mask.
Let's go ahead or let's get back to our 51.
Let's get back to visualization and run this again.
So we could see if this has been added to our 51 data set.
Scrolling down our data set, you would find that we have first of all, here at the top you have 78
instead of 77 samples as before.
If you keep scrolling, you will find this extra sample which we just added.
So this is the sample which now has been modified.
And this is what we get.
So we click on that and let's see what we get.
Okay.
So you see we have the sample, you see the green coat and then you have this mask.
Now what if we want to change this or we want to modify this edited sample such that we have red boots.
We could simply, um, get back here.
So here we have boots.
Boots?
Um, a photorealistic photo of a woman wearing a red, red colored, nice looking boot.
Boot.
Um, all red, all red.
Um, here.
All red, high resolution.
Okay, so that's fine.
Now, since we want to update that specific sample, let's, let's let's go.
Let's get back here.
Data set dot head or tail.
That's the end.
Um let's see what we get.
We have all these different samples.
Now we'll say if, if our sample um id because I think yeah, this is it here.
If our sample ID is equal, this value.
So if this is our sample ID.
There we go.
If this is our sample ID, let's take that off.
If this is our sample ID then we are going to run this.
So we're going to create a new sample and add it to our data set.
We don't need a break this time around so that's fine.
Let's run this again and see what we get.
So if this is equal that then you should run that.
So after running you could see we have this new sample with a red boots and with the green coat.
Now this is um, reproducible uh by just specifying the exact same, um, seed.
Let's take this off, get back to our visualization and run this again.
And, um, re visualize our, um, 51 data set.
We've now moved on to 79 samples.
That's from 7778, as we had previously.
Let's scroll down to this bottom and click on this.
And now you could see that this other sample has the boot changed and also the coat.
So now we have um this new sample uh which is a modification of the the previous modification actually.
So this was a previous sample.
We modified this and we got this one.
Let's reduce that a bit.
So you could see clearly.
So as we're saying we modified this and then we got this new one.
Now though you're getting a blue instead of red.
So let's modify let's modify the the way we saved our image.
We're not going to use um open CV to save our image here.
We'll instead do image which will saved is going to be the output.
That's this output uh, which is uh in the Pil format Pil appeal image.
We save this directly without um, converting to a numpy array and saving with, um, open CV.
Let's take that off and then rerun the code and we should get something now.
Better now let's we could change the, we could modify this, um, this uh seed.
So let's change the seed to say 100.
Let's run that again so we could get a different boot.
There we go.
We have this new output.
Now let's go ahead and check and rerun our, um, or refresh this.
So we, we get, um, our new, um, data set.
This is our new sample which has been created.
Um, let's go ahead and take this off.
You see, we have this boots and that's fine.
So that's it.
As we had said already, one disadvantage of this method of data augmentation is that it produces noisy
data.
So the input image, for example, which had um, only this portion to be a purse, now has this whole
portion with a purse.
So this labels this um, section wrongly.
Now what if instead of having to get to the section, run this different, um, code and then have to
get back here, refresh and get our new sample, we just make use of a plugin like this one, which
will permit us augment our data very easily.
So let's see how that works.
You see here we have error occurred during operator execution.
Um, and the reason why we're getting this is simply because we haven't selected any image.
So let's suppose that we want to augment this image.
We click on we click on a given image.
Then we get back here to this to here where we have this browse operations.
We get back there we select the plugin.
Here we select this plugin number of augmentations per sample.
Let's say we want to have five augmentations.
We want to select uh a given class.
Now the classes which we have here are only those found in the given image.
So if we picked um, this image, if we pick this image, you find that we have a vest, we have t shirt,
we have shorts, we have skin, we have shoes, we have sunglasses, we have hair and then we have accessories.
So if we get back, if we get back to our plugin, if we get back to our plugin again here among the
classes we have back hair, shoes, shorts, skin, sunglasses, t shirt, vest, uh, and accessories.
So let's say we let's get back to this.
We have five.
And then the prompt is um, let's say want to update or want to modify the vest.
So let's get back here and take out this prompt which we had already used.
So let's copy this prompt.
Um get back to visualization modify.
Let's scroll down and then modify or include this prompt right here.
So here we have it's actually a man.
So we'll say photorealistic.
Um photo of a man wearing uh let's say a blue, a blue colored, um, nice looking, um, let's say
Gucci vest.
That's fine.
So that's fine.
Um, all blue, all blue, all blue and high contrast.
So you find that instead of having to go and check manually, um, the different categories which are
found in that given image, you, you can automatically generate it from here or make use of that already
generated.
And then you specify a number of inference steps.
So 50 is a default value.
We'll set it.
We'll leave it at that.
The default guidance scale is seven.
Um we could increase the guidance scale to say eight or.
Well we have steps of.
So we take nine and then a random set of ten.
So let's execute that and then see what we get.
While still pixelating we could check out our data because obviously it's already saved.
So we scroll down.
We get to this point.
You see we have the old have the original image.
See right here.
Let's reduce this so you could see better.
We have the original image.
And then we would have this um augmented samples.
We chose five.
So that's why you have 12345 different samples.
Now the only difference you have 01234.
That's it.
So you see you see this one.
That's blue Gucci vest.
That's fine.
There we go.
So this original these are the other samples.
That's it.
We have this other one not really blue.
Then we have this one that's blue.
Um although it doesn't occupy exact same region as the original.
So that's it.
And then we have this other one that's, um, blue.
And then finally we have this other one which looks more like a, uh, what a Gucci vest would look
like that's still blue.
So that's fine.
So this is, um, some extra samples which we've added onto our data set.
Now we are going to dive into how to create our own plugin like the one we just used.
That's uh, that's actually simplifying our work and also helping others in the community, um, easily
create this or augment their own data set.
So obviously a plugin is used or could be used in many other different cases, but in our own case we're
using it for data augmentation.
Now let's get back here where we'll see how to download plugins.
So there are two plugins I'm downloading.
Although I just need this just this one.
So here uh we have this plugin which we download is situated in this GitHub repo.
And then here is his name.
So here we have in this GitHub repo the document.
Um and with this name we simply download that.
And then we get to use it as you.
You saw us use it before.
So let's get into the code.
That's this.
The code you have here is the exact same code you you would have, or you will be downloading to make
use of the plugin, as we had seen already.
So getting here, the only difference now is we have neural learn and here we have neural learn.
So that's the only difference.
So we we have this different files we we find here.
Let's increase this.
So you could see better.
So we have the the assets see the assets just contains the icon.
Then we have the init file.
We have the the git ignore file.
Well we just need we actually need just the init file.
We have we need this 51 uh yml file.
We have we need the requirements of txt file.
And then we have this python utils file.
Now let's start with the 51 Yaml.
So essentially here we just um describing our plugin and um saying what it does.
So here we have the 51 version which we're using.
We have the name Neural learn.
Um the document we have the version of our plugin.
So we could say 1.00.
That's fine.
Um, then we have description test out various augmented.
Uh, these are not argumentations.
This is going to be um, augment um segmentation data set using um stable diffusion inpainting model.
That's fine.
So that's that's the description of our plugin.
Take this off and then we have the URL that's this, um, GitHub URL right here.
And then we have different operators.
Now we're going to check this out in our init method or rather in our init file.
So we have this init.
And in this init we'll see that at the end we have the different operators which we could register.
So we're registering this SD that's standard um stable diffusion augment operator in our plugin.
Now this stable diffusion augment if we scroll up um we have this property, we have or we have this
config method.
And here we specify this name augment with standard deviation or rather with stable diffusion.
Um inpainting.
If we get back to 51 this is exact same name which matches with that.
So we are looking at how to create uh, our own plugin.
Um, essentially or most of what we'll be doing will be in this init method.
So we could dive into or let's just dive into how the plugin is created simply now in this config here
we have this operator config.
We specify the name.
We have a label, we have a description and we set dynamic to true.
Then we have the icon.
So that's in our assets icon SVG.
That's it.
And then we return this config.
So that's it for the config.
Now we have two these two methods which we are using.
That's the resolve um input this resolve input method.
And then we also have this um execute method.
We're not really using this um resolve delegation method.
When we get back to the plugin you will find that we have this, um, interface right here, which makes
our work really easy.
But this interface needs to be pre-designed.
And, uh, where we actually pre-designed this interface is in this resolve input method.
So this acts like our front end.
Let's reduce this.
So you see that clearly.
So in our resolve input method that's this resolve input method acts like our front end.
While the execute method now has our acts like our back end.
In this resolve input method we're going to create this form view.
Now we'll specify the label and the description.
So here we have augmented with stable diffusion inpainting and description.
Apply and uh apply uh stable diffusion inpainting to an image mask pair.
Getting back here we have applying stable diffusion inpainting to the image mask pair.
So that's it.
So just as in the code we had the label augmented with stable diffusion inpainting.
And here we also have augment with stable diffusion inpainting.
So that's the label.
And then the description is shown just right here.
So that's it.
We've created our form view.
The next step will be to um get all this um data from the user.
So if we get back to the code you will find that here we have this inputs.
We should specify this one.
We specify it to be an int.
So we are here we have number of augmentations per sample.
So that's number of arcs.
This is very important because we will be using this in the back end.
So you have to be careful with this.
Then the label number of augmentations per sample.
If you check this here you'll see we specify number of of um well this is it here number of augmentations
per sample.
And then we have the description, um, the number of random augmentations to apply to each sample,
which is exactly what we have here.
So that's it.
And then because it's um, an int, you have this option, you see, you, you, you put um, let's
say two or if you increase, you could increase.
You see, it takes the integers.
That's fine.
Although takes negative negative two.
So we should, um, fix this.
Or you could try to fix this and make sure it takes only positives.
Or you could take negatives.
And then you, you return an error in case it gives or you take uh, as input a negative here from the
back end.
Now for our next we have the class.
We're just going to skip this and and get to the prompt.
And then we'll get back to this.
Since this is a little more complicated as compared to all the remaining, um, inputs.
So we have the prompt.
You get back to the code.
You see here we have prompt.
Let's scroll through.
Um, all this here.
We're not going to take this into consideration for now.
So keep that aside.
We have the prompt which is a string which makes sense.
So here we have prompt um we have the label and then we have the description.
So it's required.
So we'll set that to true.
Um that's it.
Now all this normally should be.
So we should we could modify this here.
And set required.
Required and set to true okay.
So that's fine.
Um let's continue.
We have the inference or the inference steps number of inference steps.
Here you have an int and then the default value is 50.
So that's it.
We have the default value set to 50.
And then the view.
Now the view here is is different from what we had with the prompt.
And and what we also had with the int or what we had with the number of augmentations.
So this view here is is different.
Um, the difference here is that we are having a slider.
So here we have this inference steps slider which we've which we've created from here.
But it's actually a slider view.
So with the form or rather with the int here with the number of augmentations we had a field view.
Whereas here we have a slider view.
Now we're going to specify the mean value.
Obviously we have a label.
We have the mean value goes from 50 to 200 and number of steps is ten.
So we go from 50 to 200 and we go through um steps of ten.
So that's why when you, when you get here and you increase, you see you go from 50, 60, um, 70
and so on and so forth up to 200.
So that's it then?
Um, that's it.
We move to the next.
We have the gradient scale is similar to what we've just seen.
Now this a slider um again.
And then we have min value one max value 30 steps of two.
So that's it.
And the default is set to seven.
That's why if you notice here our default is set here to seven.
See slides.
Um we go through steps of two.
That's fine.
We have then the next our random seed.
So for random seed we go from 1 to 1000 with steps of one.
The default is ten.
That's why we get here.
Again you find the default to be ten okay.
So that's it.
Now getting to the class.
If you get back to the class let's get back up to the class.
Um right here.
Let's uncomment this part.
This.
Uncomment this part we have.
Well let's get back and actually uncomment this.
So we have this uncommented.
There we go.
Um here again we have all unlike the others, we actually have an enum.
So it's an enumeration instead of a string or an int.
Um and here we have this different class choices.
Now remember when you when you pick a specific image, the different classes you are shown will depend
on that image.
So here we have accessories hair necklace pants, shoes, skin vest.
Because of the the specific image we have picked.
Now let's say we pick this image here.
It doesn't contain too many classes.
So we'll take this and let's unselect this one.
Also note that we are dealing only with one with one specific um image.
So we click on this one and then you see.
You see here we have sunglasses, skin shoes, hair bodysuit, accessories.
And this is simply because once we pick a sample, we go ahead and check out all the different, um,
categories which are found in this mask.
So let's get back here and see how that's done.
So for every or for when we select a given sample you see here we have our target view.
See context view select we select one.
We we we have a given sample which we've selected.
We then say okay for a given target view.
Normally we should select only one.
So if we if we we we're going to go through this target view.
And then we break once we're done with the first one.
So once we select our target view we go through the different samples.
That's normally supposed to be one.
If we pick just one then we obtain the mask.
So for a given sample we could get the mask by by doing uh sample ground truth mask path.
With this with OpenCV we obtain the mask.
And then we now get the unique, um, choices.
So the unique or you could say we could call this unique categories.
So what we simply do is make use of Numpy's unique method which takes in the mask.
And then we convert the values we get.
Because obviously if we have say um, class zero, we have class 23 and class 38 for example, then
we want to convert this into the uh corresponding label.
So we use um id to label and we get we take in this, I convert it and then we have for example the
the background.
But because we don't want the background, um, you will notice that here we take from one um to the
end because we always start with the background.
Obviously for every image we will have the background.
So that's it.
And then we create this tuple which contains our unique choices.
The reason why we're creating this unique tuple is because of the way the drop down type works.
So it takes in this um it takes in actually tuples.
So that said uh, what we're doing here is we, we specify how we create this class choices.
That's it.
We specify a label class.
And then for all the different classes in our unique choices we are going to add a choice.
Or we're going to add a specific class into those class choices.
So now you see that it's only those which we got from the mask which we're going to add into this class
choices.
And now this is an enum.
But the values we are going to have is gotten from this class choices.
So we have class choices values.
We have the default set to skin.
And then the view is the class choices.
So it's obviously this drop down view.
So saying that the view is class choices is simply saying we want to have a drop down view as we have
seen here.
So here you see we have this drop down view unlike here where we have the slider view.
So that's it.
So we for this one we have a form view for this one.
For this one we have a drop down view for this.
Other ones we have a slider view.
We then return the property which takes in the input and the view.
So that's it.
So all these inputs we have been creating here is what we're going to have now.
That's fine.
We move ahead into the execution or in our execute method.
So in the execute method which um plays a role of our back end, we're going to take all information
from the front end which is now our resolve input method.
And then um, carry out the augmentation.
So where we have number of images per prompt, you see context params, we get the number of augmentations.
The default value is one.
So if you get back here you see we have um number of augmentation numbers.
And then for the next we have the the selected class.
You see the default value is keen.
And then here we have class choices.
So if we get to our class you see here we have class choices.
And then we get to the prompt again here we have prompt none provided um gradient scale.
The default value is seven.
The name is gradient scale number of inferences uh steps.
Um and then we also have the random seed which is by default is ten.
So that's it.
And then we have our target view.
With our target view we are able to um get a specific sample.
So the when you do when let's get out of this, when you pick um, a selected sample like this one,
we're saying that our target view um now because it's here we have two.
So our target view will be made up of this two samples.
So if we take if we take this off and then we select this, our target view is made up of only this
sample.
So.
For the sample, um, or samples in our target view, we're going to do exact same thing we had seen
already in the Colab.
So here we call on our transform sample method which takes in the sample the specific class, um, the
prompt, the number of images, um per prompt, the gradient scale, number of inference steps, and
then the random seed.
So once we create this new samples, now unlike with a colab where we just pick the first one here we
could create multiple samples.
So once we create, once we have this multiple samples for each and every sample, we are going to add
this into our data set.
So you see we add that sample in our data set.
Now the transform samples method isn't very different from what we had seen already.
But we could take a quick look at that.
So in our transform samples we have obviously we have the sample select class prompt and so on and so
forth.
Then we go ahead and generate our inputs based on um the samples file path and uh mask path.
Then from here we, we have the paint method which obviously generates or creates our edited outputs.
Then for each and every one of these edited outputs, here we have our list for each and every one of
these edited outputs.
For every edited image we are going to save that.
And then we are going to create a new sample.
So that's why we have this new samples list here.
So we create a new, um, 51 sample just exact as we had in the Colab.
And that's fine.
So we save the image, we save the image, we copy the the mask and then we create a new 51 sample.
Then once we're done with this, we return the samples.
And it's those samples which we use right here, which we use right here.
And uh, or which we make use of right here and add into our data set.
So that's it.
So that's fine.
All the other methods which we call like for example arc paints are found in our utils file.
So if we get here we have create pipeline.
We have generate inputs.
We have our paints uh and and then we have the create hash.
So that's it.
It's um that's all we need to create a plugin.
And so now we'll, uh, we'll save all this and then push this code to um, GitHub and then load or
download our plugin and make use of it.
Let's say instead our default random seed should be a hundred.
So instead of ten we want to change this to 100.
So that's the only modification we will make.
Let's get back here and say um our random seed random seed okay.
So we go from 1 to 1000 and our default is 100.
Let's scroll down to where we had the random seed.
Our default is 100.
So we save that and then we go ahead and push this to GitHub and then um download the plugin and make
use of it.
Once we push the code to GitHub we could get back here and then download the plugins.
So let's change this name.
We have neural learn.
Neural learn.
That's fine plugin name neural learn neural rendering.
And we run this cell.
And then now we could go ahead and visualize our data.
As expected we have our different samples right here.
Now note that we have 77 samples because we restarted our session and because we are working with Colab.
Each time you restart your session, all the data is deleted.
Nonetheless, if you're working locally on some cloud server, you wouldn't have this, um, data deleted.
Let's go ahead and check out our plugin.
You see here we have augmented with stable diffusion inpainting.
Um, if we get back up you would see we have this other um, plugin right here, which is actually,
um, 51 plugins.
Now this plugin permits you manage um, other plugins like um, this neural learn plugin we just created.
So if you run this you should have now the the plugin manager added to our list of plugins.
We'll need to refresh to have that.
Anyways let's get back here.
We have augmented with stable diffusion.
Remember we need to select um an image.
Um, so we could um carry out the augmentation.
So let's select this one.
And then we get back here we have augment.
That's fine.
Now notice how the random seed by default is 100 as we had modified it in the code.
So that's fine.
Um our prompts well number of of augmentations let's say ten.
Um the the class what we want to modify the t shirt.
Then we have number of inference steps set to 50.
Garden scale seed nine.
And then we have the prompt.
Well let's go ahead and copy again our prompt we just used.
So the documentation anyway we had our prompt from here.
Scroll down.
We have.
That's fine.
So we'll replace both by t shirt.
Then back to this.
Um.
And then peace out.
So anyway, let's, let's, let's just copy that for now and look at the image.
Well, so let's take off the ground truth okay.
So that's it.
Suppose we want to change this t shirt into one that is say black.
So it matches with this boots.
Let's click again on that.
So we select here.
And then we have our prompt a photorealistic photo of a man of a man wearing a black colored nice looking
t shirt t t shirt.
Um all black.
With, um, white logo.
That's fine.
Um.
High resolution.
Okay, so that's our prompt.
We go ahead and pick our class.
Um, yeah, we have t shirt.
That's fine.
So let's run this, let's increase this and then let's go ahead and execute.
Executing.
We just wait for a while and then we're visualize our data set.
So when we go ahead and browse operations you see we have this um manage plugins.
We have build plugin component build operator skeleton.
And then you should have this install plugin.
So we've already seen how to install the plugin.
Um um using the command.
So we don't need to go through this.
But if you click on this you see you're asked to specify a GitHub repo.
And then you could go ahead and install the plugin.
So it's a plugin that permits you um, easily work with other plugins.
Now if you click on this manage plugins, you see we have uh neural learn documents or, or and then
we have a description here.
So you could enable or disable the plugin.
Then for the requirements you just need to select the specific plugin you're working with.
And you could see all the um, different requirements or all the different libraries which were found
in our requirements.txt file.
So that said we could pick out, um, say this image and then now go ahead and um, augment.
So let's click on augment and then let's say one augment this say three times.
And then we change the class.
Well we have a shirt and we have belt.
We have pants.
There we go okay let's let's pick out shirt.
And then we have um.
Well, a photorealistic photo of a man wearing a click on this, or wearing a blue colored nice looking
Gucci vest.
Well, let's say a photo realistic photo of a man wearing a blue collared, nice looking Gucci vest
or blue high resolution.
Okay, so we have that now.
We could, um, pick out any random seed.
Let's say, let's say we want to pick out the random seed of, of um, 9 or 15.
That's fine.
Number of um, inference steps is seven.
That's fine.
Um, guidance scale nine.
And then let's go ahead and execute.
Once we're done with executing we could now go ahead and check out the generated data.
You can see the different um new samples which are generated.
You see it's actually um, resembling the prompts.
It's a blue shirt, though it's not very clear what's going on in this chest region.
We still have something that looks like the prompt we put in.
And then looking at the ground truth, you see all the zone.
We have that shirt, we move to the next one.
Let's take our ground truth so we could see the the output here.
This is quite fine ground truth.
We have that now.
Um, as we have said already, there's a little bit of noise around this chest region again, because
we expect the shirt to come right up like this.
Anyways, we move to the next one.
Let's take this off.
Next sample.
Well, um, we have a blue shirt, but it doesn't look, um, very realistic.
So, uh, but at least it follows, uh, or it matches with the ground truth, uh, mask.
We move to the next.
This next one.
Take off the ground.
Truth.
We have the shirt, which, um, still follows the prompt, though not very realistic, but at least,
um, we have this blue shirt.
Then we have the next one, which looks cool and also matches to some extent with the ground truth.
So that's it.
We just seen how we could generate new samples for our data set.
You could go ahead and play around with this plugin.
Um, and then see how you could retrain a model with a much larger data set.
So one good thing about this is that you could pick like this one for example.
You see we pick this one and then we go ahead augment.
And then we could say for example we want say 20 samples.
So you could increase uh, as many or you could increase the number of samples in a way you want.
Now let's see what happens when we change the or when we change the person's skin.
So let's get to the prompt.
We have, uh, a photo, a photorealistic photo of, uh, of an Indian man, Indian man, um, wearing
a red colored or.
Well, let's say we're in a blue colored.
Let's let's just say an Indian man.
So let's take this off, um, high resolution.
Okay.
Let's say a photorealistic photo of an Indian man.
High resolution, then number of inference steps 50 guidance skill nine random seed.
Let's take ten.
We have our newly generated samples.
We have a couple of them.
You see how we go from the black man to an Indian man.
And, uh, one thing you could note here is the fact that sometimes the model generates the image with,
uh, a long sleeve instead of a short sleeve.
So you could want or you may want to, um, delete some of the samples where we have the long sleeve.
So, like this one, we could delete this, delete this, delete this sample.
And this one, this one and this one.
So we're left with only, uh, when the the man is putting on a short sleeve.
Anyways, uh, once you select this, you just get back here.
You get back here.
Anyways, once you select this, you just get back here and then you type delete and you have delete
selected samples.
So we execute that with all the samples and long sleeves deleted.
Here's what we're left with.
You could notice clearly the skin and even the race change.
We've come to the end of the section.
Thank you for getting around up to this point.
If you want to continue learning, you could head over to neural AI and keep improving your deep learning
skills.
But normal may mean.
Great.
But I need.