In this lecture we are going to look at the codes implement an auto regressive linear model for time
series prediction.
Since you already know how to write linear regression Using pi storage that's not going to be much of
a challenge in this lecture.
The challenge in this lecture mostly has to do with processing the data correctly and making forecasts.
This lecture is going to walk you through a prepared call lab notebook.
Although a very good exercise which I always recommend is once you know how this is done to try and
recreate it yourself with as few references as possible as usual you can look at the title of the notebook
to determine what notebook we are currently looking at.
So let's get started.
The new thing here is after the imports we're going to start by creating a synthetic dataset.
Remember how earlier I explain why synthetic datasets are important.
They allow us to carefully study the behaviour of our model under different controlled circumstances.
So in this lecture we're going to start with a very basic time series a sine wave to start we're going
to create a sine wave with no noise just a pure sine wave with 1000 points we also plot the series so
you know what it looks like
all right.
So there's the sine wave.
Next we build our data set and this example we're going to set t equal to 10 which means that we're
going to use 10 previous timestamps to predict the next timestamp.
We'll start by populating X and Y as lists and then cast them into arrays.
Once we are done you could alternatively pre allocate these as fixed size an umpire is but I've decided
to take a more lazy approach.
So in this loop we go from zero up to the length of the series minus Big T.
Now of course you want to double check that this is correct.
So that we don't have an off by one error.
The reasoning is this.
We know that our input x will go from Little T up to Little T plus big T.
So for example zero up to 10 in terms of actual indices that will be at 0 1 2 and so on up to 9 inclusive.
So when we say zero up to 10 it doesn't actually include the index 10.
Instead the target will be at the index 10.
We know that the final index of the original series will be the final target.
That's the index Lens series minus 1 thus in order for the final target to be the length of the series
minus 1.
That means Little T must go up to the length of the series minus Big T.
If we plug in the final value of Little T we get Lens series minus T minus one plus T which is equal
to land series minus one so inside the loop we assign a little X to be the series from Little T up to
Little T plus big T then we append this to Big X next we assign a little Y which is the target to be
the series at time A little T plus big T then we append this to big Y when we're done that we cast x
and y to no higher res and prints out the shape
so as expected X is of shape and my 10 whereas Y is end by 1.
Next we build our auto regressive linear model as you know it's just an end linear
as expected at the next step is to create our loss and optimizer since this is regression we'll be using
the means squared error
next we split the data into train and test sets.
Here's another important point about training time series models you don't want to split up your data
randomly say using the cycle learn function at train test split.
This wouldn't make sense because a real forecasting model has to predict the future it can't train on
points within that future.
Thus our model trains only on the first half of the dataset and validates on the 2nd half since sideways
are periodic.
This is a non issue in the scripts but in general this is important
next we have our training function which is called full D because it does full gradient descent instead
of batch gradient descent.
This is okay for this dataset because this dataset is quite small
all right so next we call our Phil G.D. function in training is almost instant
OK.
So next we plot our loss as usual so this looks at it
next.
We're going to do a forecast using the incorrect method.
The reason I want to do this is because I want to show you what not to do.
We're also going to do this calculation manually because the general form of the code is going to be
helpful when we want to make actual forecasts as an exercise.
You might want to think about how to make this code do the same thing but more efficiently.
So first we assign the validation target variable which is just the second half of why we'll create
an empty list for the validation predictions and then populate this as we move through the data.
Next we're going to set the index variable AI which indexes x test to 0.
Next we're going to enter a loop that continues while the length of the validation predictions list
is less than the length of the validation target list.
Once they have the same length we know that we're done because we've made all the predictions to correspond
with the targets
inside the loop.
We get our model prediction P.
Now this might look weird to you.
Why not just pass in x test Saibai to the model and then assign that to P.
Well there are two problems there.
First X tests Abi does not have the right shape.
Remember that the inputs in the model must be a two dimensional array but X test itself is a two dimensional
array.
Sophie index x test.
We only get a one to array of size T.
Therefore we need to reshape the array to one by T.
That means one sample with t features we can accomplish that by calling the view function and passing
in one minus one then we get the output of the model prediction.
And that's also not the right shape because remember that in general the model returns and n by K output.
If we have N samples and K output nodes in this particular example we happen to have one sample and
one output node.
So this returns a 1 by 1 2 dimensional array therefore to get the scalar value contained within it.
We have to index the array at 0 0 finally since the output value lives in PI towards land.
We want to bring it back to Python land by calling the item function.
Next we increment AI and we append the prediction P to our validation predictions.
Next we're going to plot our predictions against the targets
so this looks very good.
Pretty much exactly correct.
But remember this is still the wrong way of forecasting.
Next we're going to do forecasting the correct way.
So if you just glance at this code so it looks almost exactly the same as what we had before we start
by grabbing our validation targets as the second half of Y and we initialize validation predictions
as an empty list.
The next line is different.
Before we just had an index i and we used I to index x test the input data.
But the problem was we shouldn't have been using the true input data to predict future values of the
time series.
Instead we'll set the variable last X to be the first input vector from here on out.
We will no longer take any new values from the actual data set.
So next we enter the loop inside the loop.
We do the same prediction as before.
Remember that we have to reshape the input to one by T.
Now this is where things change first.
After getting the model output which we'll call P I'm not going to immediately grab the value and bring
it back to Python land instead I'll do that on the next line where I appends the prediction to the list
of predictions so I index p at 0 0 and then I call dot item.
So why did I do that.
Well I still need the p value to be in PI torch land in order to make our new input on the next iteration
of the loop.
We essentially have to shift all the values of the array to the left and then add the new prediction
at the end.
Of course this entire array must be in PI torch land and not num pilot.
So that's why I kept the p variable as it was returned by the model lastly we use the PI torch Kat function
which stands for concatenate.
I take everything in the previous input x throw away the first value and then concatenate it with P..
The newest prediction.
So that's how we update the variable last X with our latest forecasted prediction.
So let's run this
and finally let's plot the predictions against the targets.
So this is very interesting.
So it appears even though we are now making the forecast using the correct method we still get a perfect
prediction.
Now remember that this time series has no noise so you might be wondering what happens if it does have
noise.
Let's go back and add noise and then run everything again
so let's scroll back up.
Let's uncommon that line
and then this good at runtime and run after
OK.
So here's the noisy sine wave
here's the other code so you see that the loss is a little higher now
lost more iterations still looks good.
So here's where things get tricky.
We're going to plot our incorrectly made forecast as you can see it appears to look pretty good.
Our forecast even appears to smooth out all the noise but remember this is the wrong way to forecast
so it should be no surprise that these results are misleading.
So let's go down to the correct way of making forecasts.
OK so now we see what happens when we do a correct forecast which is what happens in the realistic scenario.
As you can see the results are no longer as good because of all that noise.
Interestingly the model does learn that the pattern is periodic and that the true underlying function
is smooth.
So at least that's pretty encouraging.