In this lecture we are going to use an R.N. in a four time series prediction.
The previous lecturer looked at how to use an auto regressive linear model for time series prediction.
So in this lecture our main goal is to ask the question does an R.N. end do better this lecture is going
to walk you through a prepared call lab notebook.
Although a very good exercise which I always recommend is once you know how this is done to try and
recreate it yourself with as few references as possible.
As usual you can look at the title of the notebook to determine what notebook we are currently looking
at.
As you know most of this script is the same as what you saw previously.
So first we do all our usual inputs.
Next we create our data which is a sine wave with some optional noise
so here's the sine wave without noise
as with this last lecture we are going to run this notebook twice once without noise and once with noise
to see how the Arnon performs
next we're going to create our data set.
This turns our problem into a supervised learning format where we have an end by t by D input and an
end length target.
So I assume you already understand this loop since it's the same as the last script.
The only difference here is at the end we need to reshape X to be of shape end by t by D.
Where D is one in order to do this.
We can call reshape minus one at t 1 so minus one acts as a wildcard here.
So basically means that make it whatever size is necessary so that the second and third dimensions are
t and 1.
Next we're going to set the device
next we're going to build our Orient n model.
Luckily we saw this in the last lecture so this will just be a review first.
Our simple Arnett will inherit from an end module inside the constructor.
We're going to take in several arguments the number of input features the number of hidden units the
number of on end layers and the number of outputs First we assign these arguments to instance variables
D and an L as per our conventions in this chorus.
Next we instantiate the R in an object
for the input size we pass in D for the hidden size we pass in M for the number of layers we pass an
L for the none then McGarrity we pass in the real you and we also want to set the argument that's first
equal to true.
This is to make sure the R9 module uses the convention that the end dimension comes first.
Next we have our final dense layer of size m by k.
Next we have the forward function as input.
We take an x a batch of sequences of shape and by t by D.
Inside the function we start by creating the initial head and state h not a tensor of zeroes of shape.
L by end by M.
This is because we'll have an initial hidden state for each hour and then layer each sample and any
can feature.
Next we pass an X and each not into the R and end module which returns us two sets of hidden states.
We typically only want the first one since it's indexed by time rather than indexed by the hidden layer.
In this script we're building a many to 1 or an N and in this case we only need the hidden state at
the final time step in order to obtain it.
We're going to index the R.N. and output by minus 1 at the second dimension which gives us back in array
of size end by M finally we pass this end by a value into our final hidden layer which gives us an end
by K output.
Next we instantiate are simple or an N and move the parameters to the GP you.
Next we create our loss and optimizer to MSE loss and the atom optimizer
now I just want to mention this again since it's important when we do our train test split.
We don't want to just randomly pick samples across the entire time period for the validation set.
The purpose of forecasting is to predict the future and thus when you split up your data your validation
set should always be future data points.
And the train set should only contain points before that.
Next we move the data to the GP you
next we train the model.
Again this is all stuff you've seen before.
So there is no need to repeat it.
Next we plot our loss.
So that looks pretty good.
Next we do our ones that forecast.
This is almost the same as our auto regressive example except the shapes are a bit different.
The important part of this is how we shape the input and how we retrieve the output.
Since we would like the input to be of shape and by t by D we reshape it to one by T by one since and
is one and the is one when we retrieve the output which is n by K we have N equals 1 and Kagan's 1.
So we index by 0 0
so let's plot the predictions
and as expected.
This looks pretty nice but of course we know this can be misleading
so let's scroll down to the real multi-step forecast.
So here we can see that our on n with the default parameters does not perform as well.
This is a very intriguing result.
If you recall the auto regressive linear model does this perfectly and in fact we proved that we could
find the coefficients of an R2 model analytically.
So why does an orange and not perform as well.
The answer is that an hour an N has too much flexibility.
This is just like how an ANZ is more general than a CNN.
But just because in an ad is more general and more flexible than a CNN it does not mean that an ad then
will perform better than a CNN.
In fact constraining the parameters of a CNN to be shared makes us CNN better and we see the same theme
here in our two model would work best because that model actually perfectly fits the data set.
It's the correct model and aunt and model does not.
It has too much flexibility and too many parameters.
We say that it is over parameter at least although as you'll see later the flexibility of an origin
does allow it to do more powerful things that an A.R. model cannot.
Next we're going to go back and add some noise.
This dataset so that it better mimics what you might find in the real world.
So let's run all this again.
So there's the data
loss preparation is that once that forecast.
So if we look at the ones that forecast we can see it it looks pretty good.
Which is what we expect
but if we look at the multi-step forecast it looks pretty terrible.
So in fact in Orange performs worse in this scenario I should mention that sometimes when I run this
I see good results from the aren't in but generally speaking the results are unpredictable.
Sometimes a capture is periodicity and sometimes not.
So it may suggest that my tuning would be required.