In this lecture, we are going to discuss more on the topic of stationery, previously we looked at
how different kinds of Arima models might fare on the airline passengers data set.
Of course, we were just guessing.
It's not easy to tell what are the right orders to use when fitting NRMA model.
In fact, this is the case in general for machine learning in a deep learning.
For example, one question I often get from beginners is how do I choose the hyper parameters?
How do I choose the learning rate, the hidden layer size, the number of hidden units and so on?
I think today people generally have a better understanding.
But when I first started my courses, it would make people really angry that there wasn't some formula
that they could use.
Indeed, hyper parameter optimization isn't a topic for students who like simple direct in a straightforward
answers to their problems.
Usually it amounts to nothing but trial and error.
As I always say, machine learning is experimentation, not philosophy.
If you want to know whether something is going to work or not, well, then you do an experiment.
In any case, this idea will come into play later.
But for now, what I want to say is this.
For Arima, there is a way that you can scientifically and methodically choose your hyper parameters.
In our case, these are the orders PD and queue.
Note that this process is not exact and does not necessarily lead to the best answer.
However, it is statistically sound.
The first scenario I would like to consider is stationary.
As you recall, this will help us choose the order deep in our Arima model.
So this lecture will be split up into two parts, the first part will be a more beginner and practical
oriented discussion.
We're going to look at how to determine whether or not a time series is stationary in code.
This involves doing a statistical test and checking the P value.
The second part will be more advanced and it will discuss stationary in a more exact manner.
The second part is optional, so feel free to skip it if you want to jump straight to the code or if
you do not like math.
So this is the first part on the practical aspects of stationary stationary, loosely speaking, means
that the distribution of the data does not change over time.
That is, if you look at things like a mean or the variance at any point in the time series, they will
always be the same.
Looking at when this is not the case is helpful.
For example, if you see a time series trending upwards or downwards, then you know that the mean is
changing.
Therefore, the existence of any trend means that the time series is not stationary.
Furthermore, when the variance changes over time, that is also not stationary.
So if at the beginning of a time series the value only wiggles around a little bit, but then starts
to wiggle around more and more later, and that's not stationary.
We've seen this behavior in stock returns which have heteros good activity.
So one might consider stock returns to be non stationary, although it is often assumed that they are.
So don't be surprised if we treat stock returns as if they are stationary in the future.
Let's now talk about a practical issue, how can we test whether or not a time series is stationary?
Luckily, there is a well known statistical test that does exactly this.
It's called the Augmented Deqi Fuller Test or the eighth test for sure.
As we discussed earlier, one way to think of statistical tests is like an API.
We have a null hypothesis and an alternative hypothesis.
We plug our data in and we get a P value as output.
We check whether the P value is below our significant threshold.
If it is, then we reject the null hypothesis.
So really, we don't have to understand the augmented Dickie Fuller test.
We just have to know what it is for.
We have to know what the null hypothesis is and what the alternative is.
Once we know these things, we can use the test.
So what is the null hypothesis?
The null hypothesis is that the time series is non stationary.
The alternative hypothesis is that the TIME series is stationary.
So if we find a P value less than, say, five percent, then we will reject the null hypothesis and
we will say that the TIME series is stationary.
And just to bring this back to Arima, how would we use this?
Well, recall that this all has to do with the AI component of the Arima model.
We want to difference our Time series until it becomes stationary so the process will go something like
this.
First, we just have our Raw Time series.
Maybe it looks pretty stationary already.
If so, we'll do an ATF test.
If we get a P value below the significance threshold, we'll say it's stationary and so we'll then fit
the rest of the A model or equivalently Woolfenden Arima, where we say that D is equal to zero otherwise
will difference the data set.
Then we'll run our ATF test again.
Again, if we get a P value below the significance threshold, then we'll say it's stationary.
Since we differenced once we'll say that D is equal to one.
If the ATF test still does not reject the null, we might difference again.
All right, so let's move on to the second optional part of this lecture where we discuss stationary
more in depth.
So what is stationary?
Previously, we've only discussed this concept informally, but now we are ready to be more exact.
In fact, there are two kinds of stationary, strong and weak, strong stationary means that the distribution
of the random variables in your stochastic process does not change over time.
As a rough example of this.
Suppose I take an arbitrary window over some time series, then a suppose I move this window over tão
time steps strong stationary would say that the distribution over these random variables is the same
no matter what tau is.
In other words, no matter where I look in the Time series, I see the same distribution.
There is a very formal definition for a strong sense stationary, but this is definitely not necessary
to understand for this cause.
In fact, in the practical application of Time series analysis, strong sense stationary is not used
very often.
A more practical kind of stationary is weak and stationary, weak stationary looks at first and second
order statistics rather than the full distribution.
As you know, first order statistics usually corresponds to the ME second order.
Statistics corresponds to things like variance and covariance.
You already know that informal definition of weak stationary.
It's that the mean in the covariance don't change over time.
All right.
But now we're going to look at this in a more exact way.
Luckily, I think these are pretty straightforward.
If you don't find that to be the case, it's not absolutely necessary to understand what we're going
to do next.
So for the mean, it just says that the mean time T is equal to the meantime T plus tau for all tau.
That makes sense.
It means that no matter where we look, the mean is always the same.
For the second order statistics, it gets a little tricky.
It says that the auto covariance for some random variable Y at time T1 and some other random variable
Y at times too is only a function of the time difference between a T one and two.
Now, that's probably confusing, so let's think about what that means.
First of all, what is auto covariance?
Well, auto means self.
We've seen this plenty of times, auto regressive model, auto encoder and so on.
The auto covariance between Whyatt T1 and Wyatts two is really just the covariance between a Wyatts
one.
And why it's to the auto part really just means that why it's one and why a t to come from the same
time series.
Again, this is just the covariance and what is covariance?
Well, we've learned that it is the unscathed correlation.
Therefore it tells us how related to random variables are.
If they are completely unrelated, then the correlation and hence the covariance will be zero.
If they are related, that is, they move together either in the same direction or the opposite direction,
then this value will be non-zero.
If they move in the same direction, then the covariance will be greater than zero.
If they move in opposite directions, then this value will be less than zero.
So why does stationary mean that the covariance can be written as just the time difference between a
T one and two?
Well, the intuition is this T one minus two is just the distance between the two time points.
That means if I pick any two time points in the series, as long as this time difference is the same,
the covariance between these two random variables is the same.
For example, the covariance between a time one and time three is the same as the covariance between
a time three and time five.
And that's the same as the covariance between time 10 a.m. 12.
The distance between all of these is two.
In other words, the relationship between each value and the time series remains constant over time.
This actually makes a lot of sense in terms of auto regressive models.
If this relationship were to change over time, then we wouldn't be able to fit any such model.
That's why we want stationary when we fit these kinds of models.
For example, suppose our auto regressive model is way of t equal to zero point five times Y of T minus
one.
But imagine if this were only true for T equals two and not T equals three and so forth.
Then this equation doesn't work.
In order for this equation to work, this relationship has to hold for all times.
Not that this also implies that the variance remains constant over time.
If it's true that the auto covariance depends only on the time difference, then it doesn't matter what
time we pick K1.
One is just equal to zero zero.
Katsu two is just equal to zero zero.
So that's why if we see the variance change over time, as we do with volatility clustering, then we
take that as evidence that the Time series is nine stationary.
OK, so why is the concept of stationary useful?
Well, you've already learned one important reason, which is that if your Time series is not stationary,
then we cannot even use a single model to forecast.
This is because if our Time series were not stationary, then we would need a different model at each
point in time, which is clearly not useful.
Another simpler reason is this.
Recall that when we're given a time series, we would often like to compute statistics from the Time
series.
For example, what is the mean and what is the variance?
Well, if the Time series is changing over time, then it makes no sense to refer to the mean or the
variance.
This is because each point in time has a different meaning and a different variance.
That is for a non stationary time series.
The mean and variance could be functions of time.
So as an example, imagine that you wanted to compute the mean daily stock return.
In order to do that, you would have to take the daily stock return over some window of time.
That only makes sense if you believe the stock return to stationary.
So it clearly wouldn't make sense to compute something like the mean price.
Typically, when we want to compute estimates like the sample mean and the sample variance, we need
samples.
In other words, samples which all come from the same distribution.
Otherwise, it wouldn't really make sense to combine the samples to compute some statistic.
It only makes sense to take samples from different points in time if their properties do not change
over time.