In this lecture we are going to do another little demo on a time series.
But this time with a much more complicated signal it's still going to be a synthetic signal meaning
we can understand its behavior because we know the mathematical formula that represents it.
But as you'll see just one small change will make this task much harder for both the auto regressive
linear model and the aunt in this lecture is going to walk you through a prepared call lab notebook.
Although a very good exercise which I always recommend is once you know how this is done to try and
recreate it yourself with as few references as possible as usual you can look at the title of the notebook
to determine what notebook we are currently looking at.
OK.
So after the endpoints we create the time series.
As you can see it looks very similar to what we had previously but with one crucial difference this
is that we are squaring the input argument into the sine function because we square the input argument.
This means that the frequency and hence also the period of the wave changes over time.
So let's plot the time series
and we see exactly that in some parts the wave goes up and down very fast.
And in other parts it slows down
in the next block.
We build our dataset now because of my rule.
All data is the same.
Nothing about this changes so we don't need to explain this again.
Next we're going to build an auto regressive linear model.
You've all seen this before so I'm not going to explain it again.
Here's our loss and optimizer is trying to split here's a training function
and here is running the training function so let's see how this performs so it looks like the loss is
pretty high about zero point five zero point six.
This is much higher than what we had before with the normal sine wave.
Next we're going to plot the loss per iteration
All right.
So this does not look very good.
In fact it looks like the test lost does not decrease at all.
The train loss appears to decrease which is a sign that all the model is doing is over fitting without
actually matching the underlying pattern
so for our predictions.
Let's start by doing a one step forecast.
As a side note you'll see that I've taken a little shortcut here.
If you want to do is a one step forecast we've already built our input data for that.
All we need to do is call the model and pass an X test remember that the first half of X is our train
set and the second half of X is our test set
right.
So let's plot the results.
So as you can see it does not do very well.
It doesn't even get the values in the right range.
Knowing this we can deduce that the multi-step forecast is going to be even worse
again since we've already gone through this code.
I'm not going to explain it again.
So let's run this and see if we get
as you can see our suspicions are correct.
The linear model does a terrible forecast
so we're going to switch notebooks now.
Now we're looking at Pi torch non-linear sequence simple art in
so in this new notebook we're going to try and are an end model.
We're going to start with a simple Arnon and then move on to an Elysium later
the only different thing we need to do here is reshape our data so that it's three dimensional and by
t by D then we create our and then fit it and look at the results
so we can said that the vice define our simple R and then you've seen this before instantiate the model
create the loss and optimizer
meet the inputs and targets do our training
move to the GP you do full gradient descent.
All right so you can see that a loss is now better than the linear model.
Now this is a good lesson for the simple time series like a sine wave with no noise a linear model is
perfect and aunt and does worse at least with default parameters it has too much flexibility.
But now we have a signal that is much more difficult a signal that a linear model can't solve at all.
In this case the ANA does better because it actually has the flexibility to match the signal
next.
Let's plot the loss per iteration.
All right so that seems reasonable.
Next we're going to do a once that forecast
so as you can see this does a much better job.
It looks like it can match the frequency in most places even when the signal slows down significantly
in the second half.
This is despite the fact that the model never saw such a low frequency in the training set the values
are also approximately in the right range.
Now let's go to the multi-step forecast.
So as you can see this doesn't look that good but at least at the beginning the model appears to capture
the frequency quite well.
It doesn't know that in the second half the signal slows down.
So it kind of makes sense that the model isn't able to predict that.
Also it does a much better job than the linear model at getting the right range of values
finally.
Let's go to our next script where we use an alias T.M. instead of a simple in
so I'm gonna run the imports make the series plot the series build the data set.
Set the device.
Define out r and n so this uses an LSD M
instantiate the model create the loss and optimizer make the train set and test set
create the training function move data to the cheaper you train the model
writes of the lost values are still small.
That's good.
All right so the loss goes down pretty smoothly and we get a slightly better loss than the simple urn
in let's look at the ones that forecast
all right.
So I feel like though once that forecast looks a tiny bit better
and if we go to the multi-step forecast
I don't think there is any appreciable difference in the multi-step forecast.
So essentially this is pretty much the same as the simple line in.
Now why might this be.
Well don't make the mistake of thinking that unless teams are more popular and because they are more
popular it's probably because they are more powerful and therefore less teams are better and they can
do anything unless teams are better than aren't ends at finding long term dependencies.
But this doesn't mean that less teams are simply better at everything it doesn't mean that less teams
are magic now.
Although Ellis teams are good at finding long term dependencies it's important to realize that this
fact isn't hold true for arbitrarily long term dependencies.
There is a point where even Ellis teams forget of course that point is further than the simple aren't
n but it exists nonetheless.
So avoid absolute generalizations first of all.
And second.
Note that this dataset doesn't really have any long term dependencies at least for the time span of
the dataset as it exists right now.
Data from very far in the past isn't really going to help predict future values.
So in fact it makes sense that there's no real advantage to using Ellis teams for this problem.