In this lecture, we are going to discuss how to forecast with a Time series model now, of course,
you might say I already know how to do this.
I just call models that predict.
But that is not the answer because you don't actually know what's going on.
And when you call that function, the purpose of this lecture is to actually expose a common mistake,
although obviously, if you ever want to implement your own forecasting model from scratch, these ideas
will be helpful.
OK, so recall that one way to think of an auto regressive model is that it's just linear regression.
You can implement an auto regressive model yourself provided that you organize your data in the correct
way.
Let's recall what that is.
Suppose that we have a time series y one up to Y 10.
Also, suppose that we want to build an AR three model.
So P equals three, which means the number of columns in our data set is three.
Therefore, we organize our data into tables as follows.
The first row is y one way too and Y three, the corresponding target is Y for the second row is y two
y three and Y for the corresponding target is Y five.
All right.
So you get the idea.
Now, at this point in the course, we're a little more sophisticated than we were before.
We know that in addition to the training data, we want to have test data so that we can be sure that
our model actually predicts the future.
So let's say we would like the last three samples to be the test data.
All right.
So we call the first four rows, the training data, X train and Y train.
Then we call the last three rows the test data X test and we test what's next.
Then we fit our model, so we instantiate a linear regression object, then we call model Duffett X,
train Y train, then we can calculate the out of sample accuracy of our model by calling model that
score X test y test.
We can also make future predictions using models that predict X test.
Unfortunately, this is all wrong.
I repeat, do not do this.
So this is something I alluded to earlier in my lecture on the naive forecast in that lecture, I said
that one of the bad things marketers do is they write these articles about predicting stock prices with
the firms that make this huge mistake.
Well, this lecture is all about what that huge mistake is.
As you've seen for yourself after going through this section, it's very easy to get a plot with a model.
Predictions almost exactly track the true value.
How can this be when stock prices are so hard to predict?
Well, the reality is these models are doing nothing but making a prediction very close to the Niyi
forecast.
In other words, these models are basically just copying the last value.
They're not learning the underlying pattern.
The reason they look so close is because these plots typically contain so many data points that you
don't get to see them up close.
But how do these marketers get their place to look so good on the train set and the test set?
As you know, the model should never see data from the test set and therefore should have no opportunity
to copy those values.
Well, in fact, that's exactly where the mistake is if we split up our data more carefully, what we
would want to do is this.
Let's suppose again that the last three data points are for the test said these are the data points
that we want to forecast.
So that's why eight.
Why nine and why 10?
Why one up to why seven belong to the train said, well then what's the problem with defining our X
train on our X test matrices as we did before?
Well, as you can see, we are making a mistake by including true test values as inputs into the model.
This is not allowed.
To see why this is more clearly, let's look at how we would actually compute a forecast, suppose that
today is day seven.
We are trying to make a forecast for day eight, day nine and 10 since today is day seven.
We can only use way one up to seven to make predictions.
So, for example, if today is July seven, then obviously we can't use the values on a July eight,
July nine or July 10 because those days haven't happened yet.
OK, so that should be fine.
We can calculate in the usual way we had A is equal to B plus 51 times Y seven plus five, two times
Y six plus five, three times Y five.
That makes sense.
Next we have to calculate why.
Hat nine.
Now you just follow the pattern.
So you might say why had nine is equal to B plus Y one at times Y eight plus Y two times Y seven plus
five, three times Y six.
Unfortunately, this is all wrong.
Remember, we do not know why.
Ed, the best thing we can do is plug in.
We had a our prediction for day eight.
So in actuality we had nine is equal to B plus one times we had A plus five, two times Y seven plus
five, three times Y six.
We can repeat the same pattern for we had 10, which will depend on why had nine and we had eight and
the true value on day seven y seven.
So just in case this isn't clear, let's summarize what's happening and pretty much all of these blog
posts and even some paired courses that I've seen claim to show you how to predict stock prices with
Alethia jobs.
First, they make the mistake of not splitting up the data properly.
That is, they include test data as inputs into the forecast period.
This isn't necessarily wrong if you make it clear that you are only trying to predict one day ahead.
I've never seen anyone make this admission to by doing this.
All they've done is created a model that copies the previous value in a time series or very close to
it.
You've already seen in this course that this basically leads to the predictions being very close to
the true Time series.
As long as you're zoomed out far enough, they're basically indistinguishable.
And three, to bring this back to the new forecast, generally, this would be OK if you make it clear
that you are only trying to predict one day ahead.
And do you check that your forecast is better than the new forecast?
Obviously, this must be done on the out of sample data or the test set.
These models will not beat the forecast.
Note that this is obvious when you consider the API for stats models, as you recall, it works something
like this.
First, you create an Arima model.
When you call the Arima constructor, you pass in the training data, then you call model Duffett,
which takes in no arguments.
Later, when you want to get your sample predictions, you call results predict.
The arguments to this are only the start and end indices, not any data.
If you want to forecast, you call results forecast.
Note that the only arguments to this function is the number of forecasting steps.
Therefore, there is no point at which your model ever sees any data points from the test data.
Your model only sees the training data which is passed in when you instantiate the model.
So what is the lesson here?
The lesson is that whenever you see plots like these, pay special attention to the actual code.
Are these models actually predicting stock prices or are they just finding very computationally expensive
ways to copy the last value in a time series, most likely performing worse than the naive forecast?
Furthermore, we know that it's not stock prices that we care to predict, but rather stock returns.
If you've had experience with a machine learning, then, you know, it's probably not a good idea to
use prices as inputs or outputs to your model.
Machine learning models are generally not good at extrapolation.
So, for example, if the prices that you trained on were between one hundred dollars and two hundred
dollars, then maybe it'll learn something for that range of prices.
But what about in the future when your price goes beyond 200?
Your model has never seen this before and therefore it probably won't know what to do.
So be suspicious when you see marketing material like this and don't use test data in your forecast.