Talking about data processing.
This will be broken up into two parts.
The first part will be to normalize our data.
Normalization.
And then the next part will be to carry out data augmentation.
So we'll do data augmentation okay.
So first things first we would um normalize the data.
Now we're going to make use of this mean and standard deviations while carrying out normalization.
Essentially for an image let's say we take an image.
What we're going to do is we will take um, every pixel value subtract from that pixel value, uh,
a mean value, and then divide by the standard deviation value.
So that's what we're going to be doing.
Now it should be noted that this mean values were obtained from the image net data set.
So after the authors of this data set trained on on that or created or built this data set, they came
up with this mean values for the R, G and B channel.
So this is the r um r then g and then B channels.
So that's it.
So we have this value for r, this value for g and this value for b.
We will go ahead and create our function which we'll call preprocess.
So preprocess we have our preprocess function.
And this function takes in the image and or the image path and its corresponding annotation path.
So that's it.
And then because we want to obtain the image.
So we have our image here.
What we're going to do is we are going to make use of um TensorFlow's um read file method.
So we have read file and then we take in the image path.
And then once we finish reading this file we are also going to decode the file so that we obtain um
an output image which we could manipulate.
So we make use of decode um jpeg method that's after reading a file.
And then once we obtain the image we then go ahead and normalize it.
So we have image minus the mean as we had seen already divided by the standard deviation.
Now we are going to do something similar with the annotation.
So we will copy this and then paste out here.
Essentially here we have annotation annotation.
There we go.
We have annotation.
And then here we have annotation path.
Remember um the way we created our data set was such that we have the path for the image and then the
path for the annotation.
So that's it.
Let's get back to our function.
And then right here we are going to uh before we continue let's make sure we cast this.
So we have our image.
We'll cast this into a float.
So we have image um float 32.
And then we will do the same for the annotation.
Here we have annotation.
Um, we cast it we have annotation and cast it to float 32 and that's it.
So we simply return image and its corresponding annotation.
So now we living from the paths to the images and the masks.
Let's now create our data set which takes um into consideration this function.
So we have our Prep train data set which essentially takes in our train data set, shuffles the data
and then carries out this mapping.
So for every element we had in this previous train data set, what we are going to do is we are going
to pre-process it and we obtain the output images.
Let's run this two cells and then now we'll do for eye or we get an error mean not defined.
Um let's get back here and then simply define this.
So take this off run that again and define that.
So that's it.
We should should be working.
Well we're getting an error again we are told expected the unsigned int eight pass to parameter y of
op shop.
Um but instead got this um of type list instead.
Um error expected uh unsigned int but got 123 .675 of type float.
So okay.
Getting back here, we noticed that we were supposed to do this after the casting, not before the casting.
So running this now should be fine.
That's it.
And uh, for eye in prep, um, let's say train prep, train data set.
Let's take a single element and let's print this out.
So well let's say I j so let's print out I.
And gee, there we go.
As you could see, we have the image C 825 by five, 50 by three.
So that's our image.
And then we also have the corresponding mask 825 by 550 by one.
Now we want to have um this max to be 825 by 550.
So we want to take off this, um, last, um, dimension here.
So let's get back and then we squeeze that out.
We're going to have um, after this we have annotation.
I'll just put this in here.
So we have TensorFlow squeeze.
Let's take that off one one take off that last dimension.
So we'll specify the axis and that should be fine.
Let's take this off and run this again.
Or we could just check out the shape simply.
So let's print out the shape and print out the shape.
That's fine.
We could see now that we have oh let's get back up.
We can see now that we have this 825 by five, 50 by three and 825 by 550.
We will delve into the data augmentation part where we'll make use of Albumentations.
So first thing we'll create our org album and method org album ent um org album ent.
And then obviously it takes in the image and the corresponding mask and then we simply get our or obtain
our augmented image.
So there we go.
We have this transform which we are going to define shortly.
But for now we, we have this transform which takes in an image.
So we have the image and then the mask.
Mask.
Okay.
So that's it then.
Now what we're going to return is going to simply be our um, augmented image, which is going to be
converted into a tensor.
So here we have TensorFlow convert to tensor.
And then this takes in the augmented um output.
But remember that this augmented output is made of the image and the corresponding mask.
To understand why albumentations needs to take into consideration both the input image and its corresponding
mask, we could, um, consider this figure or this image and his mask, which we had already drawn
on the board.
So here let's suppose that we want to flip this.
So let's flip this, um, horizontally.
We flip this horizontally.
Let's make sure it takes back, um, exact same shape and put this out here.
You would find that if you only modify the image and not the mask, then you already have an erroneous
data set.
So let's get back here.
And then we also have to flip this tool.
Let's flip this.
And then you see now that we have this um data point which is exactly what we want.
And so that said, getting back to the code with, um, albumentations, all you need to do is to pass
in the image and the mask, and then it takes care of all the rest.
So you don't need to write any extra code to, uh, maybe say when you flip, whenever you flip the
image, you also need to flip the mask.
Albumentations does that automatically for you.
And here we have the image.
And then um, we specify the data type.
So here we have data type equal TensorFlow Float32.
So our data type is Float32.
And then we also you're going to return the because we we we returned in both the image and the and
the mask.
So we have convert to or let's let's just copy anyway.
Let's just type that out convert to tensor.
And then we have our augmented.
And then we specify that we want to take pick up the mask.
So we pick up the mask.
And then we specify a data type to be um float 32.
That's it.
So now we have our our argument method.
We could go ahead and define our transforms.
That's our Albumentations transform.
Albumentations is a Python library which permits developers or AI researchers easily um, include augmentation
in their computer vision, um, solutions.
So let's get back here.
And then we simply paste this out.
There we go.
We have this different transforms.
We have the transform for the training.
And then we have the transform for the validation.
Um, with the validation we're not going to do any transforms.
But for the training we have this random crop, uh, we specify the height and the width, which is
825 by 550, uh, with a probability of one, meaning that we are always going to have a random.
Up.
Um, we have the horizontal flip.
We have the vertical flip with a probability of 0.3.
Random rotate by 90 degrees, transpose sharpening random shadow, which are going to add on the image.
Um, and then we have this random brightness and contrast.
So that's it.
And then obviously for the validation as we've seen already it's just the resize.
So that's it.
And we now run that.
And then we have our method right here which we've just defined.
We told it is not defined.
We need to define that.
So we'll simply have um let's just create that here.
We simply have the height and then the weight.
Um, which is not going to be 825 by 550.
So it's going to be 512 by 512.
And this is because the model we shall be working with will take this, um, these kinds of inputs.
So this what this means here is simply that we are always going to take in our input.
825 by 550 image and then randomly crop parts of it to obtain.
Then um, the image which we are going to train our, the, our model on.
So that's why you have the probability of one, because we want we we don't or we always want to randomly
crop out parts of our image.
And then because we are always going to randomly crop parts of our image, we um, we don't need to
resize because it's already 512 by 512.
Now, if you take this or if you change this, let's say to 0.3, let's say, um, there's a probability
that sometimes it's not going to be randomly cropped.
Then in that case you need to resize.
So let's take this back.
We are always going to randomly crop and that should be fine.
So let's run um this cell and then the next cell and that's it.
So we now um gotten back to our argument method.
And then the next step will be to um, create our new data set.
So as we have seen already from here, we have this um, preprocess uh, pre-processing data set.
Um, that's after we've normalized from normalizing the data.
And then we now move on to our augmented data set.
So this augmented data set, before we move on we are going to repeat this for the validation.
Here we have val arg argument val arg argument.
And then this takes instead the val transform instead of the transform as in the case of our train or
of our arg argument.
So that's it.
And then once we get this, we run the cell.
Let's go ahead and try to create our train and validation data sets.
Um, you see we get in this error image must be a numpy array type.
Now that said, we need to create some other method in which we are going to specify that this or this
other methods here.
That is, argument methods and the val argument methods contain numpy operations.
Remember here we have this transform which is um essentially made up of this different, um albumentations
transforms.
And this obviously is done with numpy operations or known TensorFlow operations.
And so we need to make use of the TensorFlow's numpy function method to help us work with this numpy
functions, or with this numpy operations.
So that said we're going to define or we define some other pre-processing or augment method.
Uh it's actually going to be augment or let's call it the augment.
And then here we have image and then we have its corresponding mask.
Then in here we are going to get the augmented output or let's say yeah let's let's take the augmented
output.
Augmented output is simply going to be the same thing we had here already.
So uh, we just define a numpy function.
Um, and then in here we have the function and we give its name.
So arg arguments in this case.
And then we have the inputs which in this case is simply the image and the mask.
And then we have the output the type of the tensor of the output tensors.
In this case it's um float 32.
So all we need to do here is just specify float 32 and then here float 32.
Because we are obviously have the input image and the mask.
There we go.
So at this point we have the output.
Now what we're going to return.
What we're going to return here is going to be the ARG output um zero and arg output.
One.
So this is the image and the mask.
But because we are going to be using, um, the former method from hugging face, we have to know exactly
the, the way or the form in which, um, the former method, or rather the former model on hugging
face, um, receives its inputs.
So based on this, we are going to be able to know exactly how we are going to generate our outputs.
That's our data set outputs.
So in this case here you see it takes in this pixel values pixel values.
And take note of the shape batch by number of channels by height by width.
Now all this uh is optional.
And then we have the labels where again we have the batch by height by width.
Now if you remember, let's get back here.
If you could remember, um, get back to the code.
Okay.
So as we were saying, if you remember, um, back here, we carry out the squeezing.
Now, the reason why we actually did this was because we do not we do not want to work with a 25 by
five, 50 by one since, um, this former model takes in, um, inputs of shape, um, height by width.
So that's the reason why we had that.
And so getting back here, we have um, our outputs.
And then because we, we know that we have this dictionary that we need to pass in, what we could do
is we could simply create a dictionary and then specify the name.
So we have pixel values as we had in the documentation.
So pixel values.
And then here we have labels.
So we have pixel values and then the corresponding labels.
So that's it.
There we go.
We have our dictionary.
This should be instead of column.
There we go.
Now also remember uh as we've just seen in the documentation, it's batch by number of channels by height
by width.
But so far we've been we've been outputting batch by height, by width by number of channels.
So we need to transpose this.
That said we simply make use of TensorFlow transpose.
So we have transpose.
And then here we are going to um change this up.
So we go from our initial that's this.
Um.
Height by width by number of channels to channels by height by width.
By simply making use of the transpose method as we've seen.
Now, if this is zero and this is one, then this is two and then this is three.
Then here this is let's give the space.
This is zero.
This is one, this is two and this is three.
So here we are going to specify that we are going we are going to have 201.
And we'll explain why we have 201.
So let's close this up here.
There we go okay.
So the reason why we're having 201 is because initially we have batch by height by width by number of
channels.
And since we want to change this into batch by number of channels, by height by width, we want to
make sure that this number of channels or this, um, height goes from this position to the, this other
position that it goes from 1 to 2.
Actually, this doesn't have to take into consideration the batch.
So let's take off the batch and redo this again.
So let's take off the batch 012012 and then 012.
Okay let's take off the batch and redo it again okay.
So as we're saying we're going this height goes from position zero to position one.
And then uh width goes from position one to position two.
And the number of channels goes from position two to position zero.
So what we have at the zero position here is what was or is what we had previously in the at the second
position.
Maybe we should um, change this and put below so that it makes more sense because we're going from
what's above to what's below.
So we're going from this to this other one okay.
So that's it.
So as we're saying we, we, we have height that previously the or now what we have as channel.
What we have as channel was previously um at position two, what we have as height was previously at
position zero.
What we have as width was previously at position one.
So that's why you have 201 obeying that order.
And then we have the same output or the same level.
Then that's it.
So we just simply create our validation augment method tool.
So we have validation augment val augment.
And then we have here val arg or argument instead.
So that's val augment that's val argument and everything looks fine.
Let's run this again.
And then instead of this uh, instead of having this argument now we have augment.
So we yeah we have augment augment.
And then here we have val augment val augment.
There we go.
So we take this off and we run that.
Let's check out what we get.
Um in the output there's our let's check out our data set.
So for I in train data set let's take a single value print out I and see what we get.
See here we should have okay.
So we see we have our inputs.
Scroll to the top.
Let's get back okay.
So we have here batch see by number of channels by height by width.
And then our values have been normalized.
So that's it.
Which makes sense.
Um scrolling down again you see now we have scroll it back here.
Here we have our labels.
We have our labels.
Um, here we have our batch by 512 by 512.
So that's it.
So we have the image and the corresponding labels.
And that looks fine.
Now the batch is set to 32.
So it makes sense that we have 32 different elements when we um run this.