In this lecture we are going to do an example of simple linear regression in code.
Using pi talk in other words we are going to learn how to find the line of best fit.
This lecture is going to walk you through a prepared to call lab notebook.
Although a very good exercise which I always recommend is once you know how this is done to try and
recreate it yourself with as few references as possible as usual you can look at the title of the notebook
to determine what notebook we are currently looking at.
To start we're going to do a few imports you'll see throughout the course that these imports are what
will need pretty much every time.
So we have torch and torch type and then which is a module that contains a lot of useful stuff in pottery.
Next we have num pi and map plot lit num pies for doing linear algebra and mapping out lib is for doing
plotting.
Next we're going to generate some data to apply our model to.
Now you might ask why aren't we using real data.
Well don't get too excited.
That's what we're going to do next.
Using synthetic data is actually very important in machine learning but more on that later.
So to generate this data I'm going to set an equal to 20 data points.
Remember that our convention is to use the capital letter N to represent the total number of samples
and the capital letter D to represent the total number of features as we move on to more complex datasets
later in this course.
There will be some additional letters in our case a d equals 1.
So we don't really need to specify anything.
Next we're going to generate some random points on the x axis.
They're going to be uniformly distributed between minus five and plus five.
There are of course several ways you could do this.
So if you don't like the way I've done it you can feel free to implement your own.
And this applies to anything in this course.
So what we've done here is generated 20 random data points using the random function which outputs random
numbers between 0 and 1.
Next we multiply those numbers by 10 so that the range between 0 and 10.
Finally we subtract 5.
So the range becomes minus five to plus five.
Next we're going to generate the data points on the y axis because we're doing linear regression.
We would like these y data points to be linearly correlated with the X data points.
So to do that I'm going to say y equals zero point five times x minus one plus some Gaussian noise.
Pay close attention to these numbers.
This means that our true slope is zero point five and are true y intercept is minus one.
These are the values that our model is going to try and find.
Now you might ask why have I added Gaussian noise with mean zero and variance 1 rather than some other
kind of noise.
In fact when we have Gaussian noise the mean squared error becomes the correct lost function to use
and thus since we're using the means squared error as our last function we would also like to use Gaussian
noise centered at zero.
You're not expected to understand this during this course but when I mentioned these things people inevitably
end up asking me why this is the case.
So if you are interested in learning the answer I would recommend you look at the in-depth series of
courses where I've already covered all this material.
Next we're going to do a plot of the data.
As you can see it's sort of roughly forms a line it's a little bit hard to tell since that noise variance
is quite high compared to the variance of the data itself.
But I think it's pretty clear that the data is trending upward so that's everything for our dataset.
Next we have the PI towards stuff first we're going to define our model as promised is just one line
and a linear one one.
This means that we're creating a linear model with one input and one output next we're going to begin
the training process as you know it starts with defining a lost function object and an optimizer object.
Luckily you've seen this before.
So there are no surprises.
We're using the MSE loss and SGI the optimizer with a learning rate of zero point one.
Next we're going to transform X and Y into data types which are appropriate for Pi to work.
So we start by reshaping X and Y into n by 1 matrices next week has both X and Y into flow thirty twos
and convert them into torch sensors is in the next block.
I printed out the type of the inputs variable so you can see that it is indeed a torch tensor.
Next we have our main training loop to start we set an epochs the number of iterations of the loop to
30.
Now you might ask why did I choose 30 and not some other number like 10 or five or 1000.
Well the trick is I've ran the script before and tried different values already.
As a student you don't really see all the work that goes into choosing hyper parameters.
You only see the final result so you might assume that you can just pick any number and everything will
work out great.
This is often surprising for beginners.
In any case I discuss a lot about hyper parameters elsewhere so if you're interested in that please
ask me about it on the Q and A something else you didn't see in the code preparation lecture is that
I have a list of losses that I'm going to plot after we are done in each iteration of the loop.
We were going to store the loss in this list and at the end we're going to plot the loss per iteration.
This will let us understand whether or not we chose good hyper parameters and whether or not the training
process converge and so forth.
So it's quite useful.
Oh and just a hint for future examples.
We are pretty much always going to plot this after training so I don't want anyone asking me what is
this plot mean because it's going to be the exact same thing every time.
So don't forget it.
All right.
So next we have our training loop which we've seen before.
As I mentioned before the first step is this weird step we 0 the gradients.
And this is because behind the scenes pi torque is actually accumulating the gradient each time you
call backward.
So this will zero the gradient to prevent them from accumulating and give us the correct answer.
Next we do a forward pass to get the model predictions.
Since you've now seen how to make predictions and get the output as a num pi array you know this is
similar to that except we don't need to take the additional step of converting this into a null spire.
This is all in PI torch land so we get out a PI torch tensor that we can apply in the next step the
next step is to calculate the loss using the criterion we defined earlier.
Next we save the loss to our list of losses.
In this case we do want to take the loss from pi torch land into Python land.
So we call the function item.
You might ask why do we call the function item and not the function number pi like we do when we make
predictions.
And that's because the loss is a single Python number and not a number higher.
So when your tensor is a single number and you want to bring that number back to Python land then you
should use the item function.
Next we calculate the gradients Pi which encapsulates this in a function called backward to be called
from the lost tensor.
Finally we call optimize it step to do one step of gradient descent.
As always I like to print out useful information while the loop runs.
So here you can see I'm printing the iteration number along with the corresponding loss at each iteration
of the loop.
So if you look at these numbers they give you some sense of how training is progressing
but the real value comes from plotting the last per iteration which is what we have next.
This is the kind of curve we like to see in deep learning a steady decrease downward fast the beginning
and slow at the end.
This plot gives us more confidence that training has completed successfully.
It's not a 100 percent guarantee but this is at least one positive sign.
Next we're going to use our trained model to make predictions and plot the result.
As you recall a task for this model is to find a line of best fit.
It would be quite silly if we didn't naturally plot that line.
The next step is therefore to pass our inputs into our model which gives us the predictions in terms
of a torch tensor since we would prefer to work with an umpire raise.
We first detach the tensor and then call the NUM by function.
Now you might be wondering what happens if I do not call the attach.
We'll look at that in a moment.
The next step is to plot the data points using a scatter plot and then to plot the predicted line using
a line plot.
As you can see from the result our line is in fact the line of best fit.
Next we look at what will happen if you do not call the detach function as you can see.
We get an error it says can't call them PI on variable that requires grad use var dot detach that num
pi instead.
Which is exactly what we did
however this error message gives us a hint for an alternative way to make predictions.
We can see that it has something to do with gradients
in the next block of code.
We can instruct pi to not to compute gradients.
We use the context manager with torch dot.
No grad inside the block we make our prediction.
But if you notice there is no call to the detach function as you can see the output is printed correctly
without any error.
The last thing we would like to do in this script is to inspect to the parameters of the model to check
if they're close to the true values.
As you recall these are the slope and intercept of the line in machine learning.
We call this the weight and bias in order to get these values we call model dot weight dot data dot
num pi and model dot bias dot data dot num pi intuitively you can expect that model that way and model
that bias store the weight and bias as some special pi torch tensor variable.
However we would like to bring these back to num piloted so in order to do that we call data dot num
pi.
You can imagine that the reason we treat these differently from the model's predictions is because they
are model parameters rather than data inputs or outputs.
If we look at the values we can glean a few key pieces of information.
First the obvious thing we want to check are they close to the true values we get zero point four six
in minus one point two.
So it appears that they are one interesting thing about these is that w seems to be a two dimensional
array whereas B seems to be a one dimensional array.
You might ask why do we need a 2D array and a 1 b array to store single numbers.
In fact we don't but we need our model to be flexible enough to handle different scenarios.
These scenarios are what you'll learn about in the later lectures the final thing I want to talk about
in this lecture is what is the point of using synthetic data instead of real data.
This is going to come up once or twice in this course.
So it's good to discuss this early and often.
One of the most important questions we can ask is Does our model actually do what we think it does.
As a beginner taking a course your automatic reflex is to make a guess or to think of the problem philosophically
but this is not a philosophy course.
It is a computer science course.
We don't discuss maybes in this course it's binary yes or no it works or doesn't.
So synthetic data is very useful in that regard.
I can create synthetic data such that if my model does what I say it does then it will find the pattern
in the synthetic data that I set it up to find if it can.
Then my assertion is wrong doing things this way allows us to test the strengths and weaknesses of our
models and to help us understand what they can and cannot do.