That's now generate this density maps from the Shanghai Tech dataset.
The Shanghai tech dataset is made of images which are several hundred and 68 by 1024.
So that's why we define this in X and in Y as such.
Then we have our X and Y, which are the dimensions of our output, which happen to be the density maps
we've seen already in the slides.
So we are going to subsample this from 768 to 9 six and then this from 1024 to 128.
Now dividing this by eight gives us this and then dividing this by eight gives us this.
So it's been sub sampled by the same factor.
So we have this sub sampling factor which will call or which we have as in x divided by our x or in
Y, divided by our Y.
Either way it works.
So we'll take this off.
And now we go ahead to define the Gaussian distribution, which we saw in the slides.
This is defined quite simply.
We have the mean of the distribution we have as standard deviation, which we define as sigma, and
then we set it to this value of ten by default.
Then we have the formula which we saw already, which is this we shall take this separate this portion.
We have the first part, which is one divided by the square root of two pi sigma squared.
So this is the first part and then the part which takes in the X is this E to the power of negative
half times x minus mu divided by sigma squared.
So basically that's it.
Then we add this first dimension right here.
Then the next thing we do is now to define a function which generates the density map.
So there we go.
We have this get density map, Gaussian function, which takes the image, It takes in all the points
in that image which contain heads or the positions where we have heads.
And then we have this sigma value right here, which is actually the Gaussian radius.
So let's take this off.
We have this as the Gaussian radius set.
It's a default value of to take this off.
That's fine.
Now the role of this Gaussian radius is to determine the number of points which are used in generating
for each of the heads its own local Gaussian map.
Now we take this right here.
We'll see that if we have this pixel, that's let's pick this pixel in the center which contains the
head.
We see that all these different points were used to generate a Gaussian map.
If we increase the Gaussian radius when we have this occupying maybe the whole of this map.
Now, this is a hyper parameter which is used to.
Define how far each Gaussian map for each head extends.
So like in this case, you see that this is a radius like this distance from this point to this.
So this pixel, if you could actually count the number of pixels, would have one, two, three, four,
five, six, seven, eight, nine, ten, 11.
So we'll see that we have 11.
So our Gaussian radius in this case is actually ten, not 11.
So in our gotcha radius in this case is say ten, and that means that this ten points contribute in
generating this Gaussian map for this head.
Now if our Gaussian register is five, it means this patch we have right here is going to be smaller.
So we're going to have just one, two, three, four, five points.
So this blue paths right here will come right to this.
This is going to become smaller than what we have right here.
Now, if you increase this radius to say 20 instead of any year, we see that we're going to extend
right up to somewhere around this.
So we're going to have something like this.
So that's actually the concept of the Gaussian radius.
And it's a hyper parameter which you could actually turn to see whether or how it affects the model.
And also in some cases, you may make this hyper parameter adaptive depending on how well it behaves
or better still, how well it affects the model's predictions.
So that's fine.
We have the width and then the height, which is this.
We have this as a weight and then this is the height of the image.
The number of ground points is basically the length of this points right here.
Now, this points, as we said earlier, is simply all the positions in the image, which is this not
in this all the positions in this image right here which contain the heads.
Now, in order to bring this points into the reference of this density map, we shall divide all these
points by this factor right here.
So in this case, we're going to define divide all these points by eight so that we'll leave from this
refrains to this new, smaller refrains.
We now continue with our Gaussian map generation, and we shall repeat that for each of the points that's
for each of the heads we have in the image.
So that's it.
We see here we have the points.
So we selected or we pick the first point.
For example, we convert this into an integer.
Now we get the x coordinate and the y coordinate of the points, or better still, the h coordinate
and then the W current of the points, because this is the if we have an image like this one right here,
this is the origin, we have the h axis and then we have the width axis.
So this is the height and then this is the width.
Now this point, for example, is gotten based on the distance from this origin to this given height.
And then the distance from this origin to a given width right here to look at this point.
Now, the reason why we have the minimum of the height, the maximum or the height minus one, and then
the width minus one is actually because if this point happens to fall exactly at the last pixel h of
the image or.
Last pixel w of the image.
Then we consider that it's located at a pixel before its actual position.
Now once we have this, we go straight into the generation of Gaussian map using the Gaussian distribution,
which we've defined already, which is this.
Now let's look at this.
Firstly, we have this Gaussian radius, so we have this lean space.
Now let's do something like this.
Let's print out MP dot in space of say minus four and then four.
So we're suppose not a question.
Radius is four.
So in that case we have that and then we have now the Gaussian, or rather we have four times two plus
one.
Let's take this off.
Run this.
You see, we're going from -4 to 4.
And the reason why we have this specify the number of points we shall consider.
You see that?
Yeah.
We actually have nine points.
One, two, three, four, five, six, seven, eight, nine.
Now, without this, we'll have much more points, which we don't actually want to work with.
We want to control the number of points we're working with.
So let's let's have this four times two plus one.
If we modify this, let's take this out, for example.
You see we have now eight points.
And now my plan this see?
That's fine now.
And that's exactly what we do right here.
We're just putting up all those points we're going to use to generate our ocean map.
Now, once this is done, we generate a Gaussian distribution from all these points.
So let's take this, right?
Yeah.
We now say, for example, we have the Gaussian distribution of this run that you see that we have now,
this Gaussian distribution right here.
Notice that towards the center we have the highest values.
We see how we go from this one, two, three, four, five.
It actually picks at this center, that's at the mean and then it starts to drop back again.
And then this is symmetrical because as we had here, we have our mean to be zero and then sigma ten.
So obviously this is going to be symmetrical.
So we look at this, you notice that this is exactly equals this, this exactly equals this, this exactly
equals this.
And then this is actually equals this.
Now, next thing we could do is also to modify the sigma right here.
So we have this, right.
Yeah.
Says sigma equals a let's say three.
We have that.
So we see now that we have this also again, we have this highest value .13, it starts to drop, drop,
drop, drop.
Now, if we increase this values, say we take 40 values for the for the run it again.
You see that these values now become very, very small.
So this is practically zero.
And now if you go to a center like around this, you see okay, yeah.
Like let's find that maximum value, 1.2, three, two.
Yeah, this is it right here.
So this is the center right here.
So we have 1.3, two times ten, a negative one.
That's actually 0.13.
Now, this values which are towards this end right here, you see that there are practically zero.
Now, let's put this out so we have something like this.
Then we have this.
Right?
Yeah.
When sigma is equals three.
Notice how three steps we take the three values close to this peak value.
You see, they are of reasonable magnitude.
Yeah.
We have, for example, this, this and this one.
But as from this, like let's pinpoint this two as from this going downwards and then this going in
this way.
You see that the values become very, very small, up to the point where we get to the zero right here.
Now let's modify the value for sigma to say two and now see how this is also going to make the even
this third value right here become also very small.
So put that to to run it again.
Now let's check this out.
You see that this now goes to practically 0.2.
Now, this is at 0.17.
And then from this, now we have 0.1 to 0.06 and so on and so forth.
So unlike previously where we went up to -40, now we even go up to -88 to show that this draw we had
previously now even goes faster.
So instead of having that, we now have someone like this.
So that's it.
Now, let's take it back to this.
We see, as we had seen previously, if our sigma is somewhere like this, let's say this is sigma or
standard deviation of three.
Now four standard deviation drops to two.
So it means we are on this or rather drops to two.
So it means we're going to have a new SAE.
We're going to have something like this now, which is let's draw it like that.
Okay, So we're going to have something like this now.
So it means that in this case now, only very few neighbors around the mean get to have reasonable values.
And then if we were to modify this to ten, you see that instead of having this kind of curve, we're
going to have something like this now.
So it's going to be something like this.
And this is because in this case, we're extending our standard deviation.
Let's take a value of five.
So we have.
So we match up with those values right here.
So let's say six, for example.
So now our standard deviation is six.
So we see that it's going to be something like this and now this peak value that is the highest value
is going to be reduced compared to the other highest values we had because now many of the values are
shared with the neighbors.
So that's it.
Now, the next thing we'll look at is the fact that the question distribution we're trying to produce
isn't actually a one dimensional Gaussian distribution where we have this input X and then we have this
output.
In this case, we actually have two points.
So, all right, we have a point which takes two values, which we could say H and W, And so in this
case it means we have something like this.
We have.
Instead of just having 1ch we have now H and W.
So if we consider this to be the let's take this to the H axis, So let's do this.
We have this as a H axis and then this as the W axis.
Now in this case, instead of having just some two dimensional plot, Yeah, this is meant to be a three
dimensional plot.
So we're going to have let's take this we're going to have something like this.
And this is such that for every H and W, we have it's probability distribution.
And that is Now why, when you look at the the plot of the density quotient map, you have something
like this, you have this, you have this, you have this, and so on and so forth.
And that's because this is like the X and then the Y, and then we have P of x, y.
So instead of having P of X as we define it, we now have P of x, y.
But now note that we are going to actually approximate this.
Multivariate Gaussian distribution to be just simply p of X.
Times.
PFI.
So we suppose that p of x y is the same as p of x times p, f y.
And that's why I write.
Yeah, we have this non pi the multiplier where we take the x and then we have the Y.
Now to obtain this Y that's supposed to take the H and then that is P of H and then times p of W.
To obtain this, we simply find the transpose of what we got right here.
So that's how we obtain this.
And we obtain this Gaussian map right here.
So unlike previously where we just had a vector of points, now we actually have a matrix.
So let's take this, let's reduce this back to four.
Have this back as far.
Run this again.
See, we have this.
Let's let this to be g.
So we have this as g is equals this.
That's fine.
We now say print out the multiply of g, comma, g transpose.
Run that.
And there we go.
So instead of having this, let's print out a shape.
So instead of having this nine dimensional vector for the Gaussian distribution, now we have this matrix
right here, and soon we shall plot out this matrix.
So you see exactly how it looks like when I moved to the next part.
Now let's suppose we have this Gaussian radius of far right here, just as what we did here.
So this gave us this nine by nine matrix for the Gaussian map, which we calculate right here.
Now, the use of this term is to normalize this Gaussian map, because depending on the number of points
we decide to work with right here.
Women have a situation where the discussion map doesn't even sum up to one.
So what we do is we take the Gaussian map and then divide by the sum of the goods and my values to make
sure this Gaussian map has all those values.
Summing up to one.
Let's take this as an example where instead of having this, we have some so we print out and some run
it.
You see, we don't have exactly one, but if we divide this, that is if we take the sum as this.
Take this from this.
We divide by itself.
Now let's have the emperor multiply, which is the Gucci map itself.
We run it.
And then we have the sum of all this.
Anyway, let's separate this.
So let's suppose we have this Gaussian map, which is equals this.
And then we have this MP that some of the Gaussian map right here.
So we have our Gaussian map.
We now go?
I had to print out the Goshen map.
Divided by the sum of its values.
So there we go.
We run this and then we'll define a normalized quotient map.
So this is actually a normalized
Gaussian map, which is this, and then we print it out.
So we have the print.
Okay, that's fine.
We know that some of this normalized Gaussian map C we have is actually one because we've taken a regression
map and then divide it by the sum of all the normalization map or the sum of all the map values.
So that's what exactly what we do right here to normalize how good map.
Now, once we have this, we are trying to create a patch on the output image on the output map.
So what we're going to do is from this nine by nine matrix, which is this Gaussian map and the case
where we're supposed to Gaussian radiuses for we're going to have nine points, four plus two plus one,
that's nine.
We saw that already.
So we're going to have a nine by nine matrix for the Gaussian map, which we could represent as this
where we have the origin right here, we have nine steps to the right and then nine steps downwards.
So this forms are nine by nine matrix, which is this square right here.
And that's why, as you could see here, we define x left to be zero.
So for the other left position, our x coordinate is at zero.
That's this.
See our x coordinate right here, which is because this is our x axis, this is what we're supposed
to be our x axis.
So this x coordinate right here is zero.
And the left position of this square and then for the X right is the Gaussian map shape one.
Now this question, map shape one is the is nine.
So our X right is nine, which is normal because from moving this way at this point, our this position,
our x coordinate is nine and then we'll do the same for our Y up.
Our y up is actually this because this is the upper part of our square.
So this y coordinate because our y starts from zero and goes down to nine, this y coordinate is zero.
That's why right here we have zero.
And then for our Y down, which is this is actually equals nine.
So that's how we obtain this.
Now supposing the points we've selected is at say, 3624.
So if it goes 36 steps.
Yeah.
So just for go, let's take this.
36.
And then we go downward like this.
24.
So we have the point 3624 we're working with now, if we have this point 3624 now, we'll see exactly
why we need to do all this right here.
And in this case, if we compare this 0.1, point one in this case is 24.
So if we compare this h coordinate as a height coordinate with the Gaussian radius, which happens to
be four if we compare it and we see that this 24 is greater than this, we don't have to get into this.
The same for the 36.
So we compared 36 is not less than four.
So we don't have to get into this.
And then we compare the 24 plus four with the width of the density map.
In this case, it's not greater than W, So we don't have to get into this because we're supposedly
we're having a nine and six by 120, 128 density map 128 where we have the height to be nine six and
then the weight to be 128.
And so as we could see, the 24 plus four is not greater than one and 28.
So we don't have to get into this again.
For this we have the 36 plus four is not greater than 96, so we don't have to get into that.
So all this for now isn't very useful because this patch we have is actually well fitted in our density
map.
And so in that case, if we have this density map, which initially was made of only zeros, as we have
defined initially, as you could see right here, what we do is we take the patch and then we simply
add it up or stamp it on the density map, as we could see right here.
So we take this and then put it up on this density map so that all the values it gets now, the values
we had seen previously and now occupied, that is all the values which correspond to the position with
respect to this point right here, because we have the point.
3624 That's supposing we had the center.
What we do now.
We have the center.
We also have the center.
We now just take this and then feed it such that this center matches up with the center on the density
map right here.
And so to locate this map, this patch on the density map, we have point zero minus the Gaussian radius.
In this case, that happens to be 24.
So we have 24 minus four.
So we find out this position right here up to let's ignore this max of zero comparing zero and this
value for now, because as we have seen, this patch actually fits well in the density map.
So we wouldn't have any problem.
So we ignore the max, we just focusing on this.
So as we've seen, we have 24 minus four, so we add a point 20.
Now we look at this 20 with respect to the height.
So and then up to let's ignore again the minimum and let's focus only on this up to point zero plus
the Gaussian radius plus one.
Now point zero again is 24, 24 plus four plus one is 29.
So we're going from 20 to 29.
That's for the H axis.
So that's how we look at this.
And then for the W axis, we're going from point one minus the Gaussian radius.
Now 0.1 is 36, so we have 36 minus four gives us 32.
So instead of having 36, as we are seeing in the center, we start at 32 and then we go up to 32 plus
nine.
So 32 plus nine should be 41.
So that's how we look at these positions where we're going to take this Gaussian map we had generated
right here and then patch it up on the density map.
So as we've seen, once we look at these positions, all we do is just to simply add it.
As you could see right here, we so we take the density, the Gaussian map and then added it on the
density map, which initially was a value which had values of zero initially.
Now, once we have done this, we simply repeat for all the points and see now that that's how we generate
all other points like this start is a bit bigger.
We have points like this, we have have half points close to this, so we draw them like this.
Now notice that this region is where we have this intersections.
Now we create points with higher values or relatively higher values compared to all of the points where
maybe there's some there's so many spaces.
So this translates to the fact that when are many people around the same region, those values have
higher magnitudes as compared to other regions where the people are actually spaced out.
And once we have this, we will notice the density map that this region is R.P.M. yellow ish as compared
to the other regions.
Now let's suppose excretion where this conditions are verified, that is the edge conditions does supposing
we have this patch but which falls at a point.
For example, let's say let's maintain a 36 but we're following other points.
Let's make this small So we're going to if we maintain 36, then we go right up towards the end.
So if we go to the end, we'll be at say 904.
So suppose we have the point 94, let's have 94.
So we have 36, 94 right here.
So instead of 36, 24.
So see now we have 3694.
In this case, if we get into this, we'll see that everything is okay.
We have our point one, which is 94, which is greater than the Gaussian radius.
So we don't have to get into this.
Our 36 is greater than the radius.
We don't have to get into this.
But then the point one, which is 904 plus the question radius, which is four, is greater than the
width, which is 96, nine is 90, 94 plus four is 98, which is greater than nine and six.
Even in the case where was 92?
We have 92 plus the four will give us 96, which will be equal, as we could notice.
Here is equals.
This was strictly strictly.
This was strictly.
This isn't strictly.
So in this case, we will see that we have to get into this.
And the reason why we get into this is because if we are at a point, say, 94, as we are on this,
that is 694, we add this, it's impossible for us to draw this box where we go four steps below and
then four steps above, like in this case where we had this, we had this one pick zero when four steps
below, when four steps above, four steps to the left, four steps to the right.
Yeah.
We can't move freely like that because we had nine forces left just to step the remaining steps out
of the density map.
And so that's why we need to get into this so that we readjust this wide down position right here.
Now, we made a slight error.
This isn't this way.
This is actually 9436 because.
We decided to fix this, hide this width, and then we walk it in the height direction.
So this is 904 since we have the same sort of height width for a point.
So this is how we look at points.
So yeah, we have nine for 36 instead.
So we'll get back to this to fix this problem right here.
What we do is we define our wide down so that instead of having to go from 0 to 9 and 0 to 9, we're
going to.
Crap.
This I just had ten points so I would go to from 0 to 9 and instead 0 to 9.
Now we instead go from, say, in this case where we have up to nine of four, we go from say, zero
to a value like, say seven such that we now show that going from 0 to 7 and then yeah, from 0 to 9.
So we're going to have a nine by seven matrix now will permit us.
So nine by seven instead of nine by nine will permit us patch this up on the density map without having
any error.
And so to obtain this value of seven, basically that's all we do.
We have the, the shape nine minus the radius plus the 0.94 minus the height, minus one.
So with that, we've seen how to deal with this edge values, values which are around this edges around
those edges.
And all this.
Now, once we're done with this, we also make sure that the minimum height value, because we have
in this density map, the way it's structured, is such that we have the let's call this density map.
DM So we have the DM which takes in Y mean or rather H mean takes an H mean to H max and then w min
to W max so that we could come up with this square as we've seen right here.
Now in some cases we have h mean which is negative or H max which is greater than the height of our
density map.
So that's why we have this year.
Well, we try to take in the case where we have a negative value, we're going to pick the value zero,
and then the case where the value is greater than the density map's height, we're going to pick instead
of density map's height so that we don't go out of the density map.
And it's the same for the width dimension.
Now, after converting some of the edge points or after reducing the edge Gaussian map shapes in order
to fit in the density maps, we now have to re normalize our density map.
So we have this where we take the density map and then divide it by or rather the density map now is
itself divided by the sum of the density map, divided by the length of the points or by the number
of points we have in our data set for this particular density map.
So once we have this now we can return the density map and then we could go ahead and test the generation
of our density maps.
Now let's look at this.
We have the image we're going to work with or the photo for the images.
We have the photo for the maps.
We pick out an image.
Image one.
We have this particular image right here which we've selected.
We use a load image from TensorFlow or image to array from TensorFlow, and now we normalize this image.
We should put in this variable X and then we could show this.
So let's comment this part and then let's take this off.
We need to plot show run that.
Okay, so this is the image in equation C is goes from 0 to 768 in the high dimension.
And then the width dimension goes from 0 to 1024.
We had defined this previously right here.
Now, once we have this, we now go into the map itself.
So let's take this.
And to load the map information, we use sci fi and its load mat function.
Now this load map function is capable of taking this files with extension the map and then extracting
this information from it.
So once we have this, we could print out this map.
So let's print out a map.
For now, again, let's uncomment here.
We'll print out a map.
And that's fine.
So we'll look at those values.
We see that there's actually a dictionary which has several keys and values.
We have the header is value.
We have the platform key and is value created on its value.
Then we now have the information we actually need.
So this is the image info and this is the array one to work with.
So we see that this is actually the exact information we want to work with because we're interested
in this point.
As we've seen here, we just basically extract the points from our.
Mat file, which is this.
So that's why you see it right here.
We specify this.
Image info CBS specified as an image info.
So we get that array.
Now let's plot out.
Let's print out the image info.
So if we print this out, we have matte of image info.
Now run.
That is, you'll notice that we don't have this other parts of our dictionary.
Now.
We only have this again here to obtain the simple array which is made of just the points.
So we simply have this right here.
So we have to pick out this position.
So if we have this see, we now have just this which now contains all our points on which we could pass
into this follow up right here.
Now, that's what we have.
We now need to convert this points into the new dimension.
Recall that the images come as 900 and those right is 678 or 68 by 1024.
So we need to convert this into nine is six by 128.
So for every point to do this conversion, we need to divide, for example, this 32 by a certain factor.
The factor happens to be eight because to live from this to 96, we need to divide by eight and from
1024 to 128, we need divided by eight.
So that factor happens to be eight, and that's exactly what we had to find previously, the subsampling
factor.
So instead of having just the points, we subsample them.
Oh, subsampling factor.
So that's it.
And now we're ready to pass this points into this get density map Goshen function.
So that's it.
We now take this off and then we plot our density map.
So take this.
We have that.
Notice how this initially had values up to 10,022.
But now we have to modify the search that it is restrained from going above 128.
Let's let's test this now.
Run this.
You see, if we take the point where we had okay, let's print out those points instead.
We have points.
Take this off.
That's fine.
So now we see we go up to 119 and so on and so forth.
127 This should be that 10,022 So that's it.
Now, once we have this, we have our image and then we have this to say, God, okay, so let's modify
this now.
Instead of Sigma, we should have the Gaussian radius question radius.
So that's it.
Now let's take off this points.
Running this.
Now we obtain this Gaussian map right here or this density map right here made of this different Gaussian
maps.
As we could see, we have this image.
Notice that this portion where they are no heads.
This is empty.
Then that's it.
We could also test with an image.
We had the slides.
So run that, run this out of put a break.
So let's take this off.
So we have that.
Let's take this off now.
Run again.
And we find so this the image we had with this density map from the slides, we could go ahead and modify
the ghosh and radius.
Let's take a radius of, say, 50.
You see that?
This is what we opt.
Now let's reduce this to 20.
So you see how it changes slightly.
See, reduce this to ten C So that's basically it because for every point now we have less values compared
to when we had, say, 50.
With 50, we have 100 values to represent just one points.
That's why the Gaussian map looks like this.
Now let's put this back to say a value of ten.
So that's it.
Now let's modify this sigma value.
That's the standard deviation.
Let's take the standard deviation to be, for example, to run this.
And there we go.
So we see that we just the standard deviation makes this more circular, since points only points very
close to the actual points have reasonable values.
We then conclude with a data generator which takes in the images, the maps by size sub sampling factor.
Now from this we're basically repeating what we've done already, right?
Yeah, well, we process the image and then we have our values or we append this into the x list.
So basically we have this image which will happen to X, and then we have the density map, which will
also happen to Y, then we convert this to tensor and then that's the output.
So that's it.
We now have the train images which will define the train maps.
Then finally we have our train generator right here.