In this lecture we are going to look at the code to demonstrate uncertainty estimation Using pi to which
this lecture is going to walk you through a prepared code lab notebook.
Although a very good exercise which I always recommend is once you know how this is done to try and
recreate it yourself with as few references as possible as usual you can look at the title of the notebook
to determine what notebook we are currently looking at.
So let's begin by writing a function that generates data from it known synthetic distribution.
This might seem weird but in fact this is exactly what we do when we run an experiment or do data collection.
It's like we have some function called generate batch and that function is a black box which involves
whatever data generating process happens to result in the data you're measuring.
Of course the difference is that you don't ever know the true distribution of your data nor the underlying
data generating process.
And that's why we need machine learning.
But for this example we do know the true distribution and the data generating process and hopefully
that will give you some insight on what machine learning is trying to do.
So what does this function look like.
First we generate a bunch of random points for the input x from the uniform distribution between minus
five and plus five.
Next we write an expression for the standard deviation as a function of x.
As you can see it's a linear function of x.
So when x is small the standard deviation is small and when X is large the standard deviation is large.
Next we get the target we assigned the target to be the cosine of X minus some linear function of x
plus Gaussian noise center at zero with a standard deviation of SD.
Then we return x and y
if we plot the data you should be able to recognize each of these elements.
First we can see the periodic component due to the cosine.
That's why this thing appears like a wave.
Second we can see that a trend downward.
Thanks to the linear component.
Finally we see hetero skid as this city.
The noise seems to increase as X increases.
In fact we know that the noise standard deviation increases linearly with X.
Importantly we can see here that given the same X it's possible to get many different values for the
target.
Next we create our model.
This is a pretty basic custom model which contains two simple antennas.
The first CNN will be used to predicts the mean of the output and the second and then will be used to
predict the log variance
in the forward function.
You can see how this is kind of the opposite of a recommender system.
A recommender system takes in two inputs a user and an item and returns one output a predicted rating
on the other hand.
This takes in one input and produces two outputs.
Next we instantiate the model and then we create our custom laws.
So in this function we'll stick with the convention of calling the first argument outputs and the second
argument targets inside the function.
We know that outputs is a tuple with two components.
So we take the first component and assign that to a variable called Mew.
We take the second component exponential rate and assign that to a variable called V.
Next we calculate the coefficient term of the Gaussian distribution.
That's the log of the square root of two pi times V.
Next we calculate the exponent term of the Gaussian distribution.
That's zero point five divided by three times the square difference between the targets and Mew.
Lastly we add the coefficient term and the exponent term together and take the mean.
Next we have our optimizer and our training function.
So this is nothing really new.
All right.
So here's our last per iteration.
There should seem a little strange since what we're used to when we look at a loss per iteration is
that the loss usually has a steady decrease downward.
This on the other hand seems to be very noisy.
So what is the cause of all this noise.
Well remember that our data itself has noise and the loss function is the negative log likelihood.
Remember that the loss doesn't only contain just the squared error anymore.
So we shouldn't expect the loss to go down to zero since that would mean the variance is zero which
we know is it true.
Next we plot the model predictions in order to do this.
We're going to generate a large batch of 1000 twenty four points after doing a scatter plot.
We convert these points into torch is reshape them and pass them through our model.
The model returns two things are y hats and the log variance.
Since our model outputs the log of the variance we can exponential rate it to get the variance but that's
not actually what we want.
If you recall the variance doesn't have the same units as the original random variable so it doesn't
make sense to plot them together.
Rather we would like to plot the standard deviation which is the square root of the variance equivalently.
We can just divide by 2 before taking the exponent since the square root is the same thing as a power
1 1/2.
Next we plot our prediction along with the corresponding uncertainty on top of our scatter plot of the
data in order for a plot to look like it should we have the sort the data points first.
This is because map plot lib joins the points you pass in by a line in whatever order you pass them
in we can get the sword index by using the ARG sword function.
Next we plot X versus Y.
This should follow the center of the data points for each X.
Next we use the fill between function to draw a transparent band with width equal to the standard deviation
so that's why we have y hat minus SD and then y hat plus SD.
So here's the plot and as you can see the band increases as we go from left to right and our model accurately
predicts that the variance should increase as X increases.