In this lecture, we are going to continue our discussion on how to choose a rhema hyper parameters,
this lecture will focus on the key parameter, which is for the moving average.
It's kind of interesting, although entirely inconsequential, that we are discussing these backwards
in terms of the order we introduced each component of the Arima model.
It turns out that in the Arima model, it makes the most sense to discuss the auto regressive part first,
then the moving average and then the integrated part.
However, in terms of hyper parameters, it's easiest to understand stationary first, then the Akef,
which is the topic of this lecture, and then the pickoff, which will be the topic of the next lecture.
In any case, let's continue.
So this lecture is all about the akef, which it turns out will help us determine the key parameter
in the Yarema model.
So what is the ATF?
Akef stands for auto correlation function.
Note that this is also known as the telegram.
But before we talk about the auto correlation function, we have to first understand what the auto correlation
is to begin with.
Like the covariance and auto covariance, the auto correlation is similarly defined.
You will understand this if you didn't skip the second part of the stationary lecture, in case you
did, let's recap.
As you know, the covariance is defined as this expected value between any two random variables.
The auto covariance is simply the covariance between two specific random variables selected from two
possibly different time points in a time series.
That is, the auto part simply means that the two random variables came from the same time series.
Otherwise, it is still just a covariance auto means self.
So you can hopefully understand the semantics behind this terminology.
So as mentioned, auto correlation as similarly defined recall that correlation is nice because it's
a scaled version of the covariance.
As such, since we can always expect the correlation to be between minus one and plus one, we can expect
the same of the auto correlation.
Note that in some fields outside of time series analyses such as engineering, the terms auto covariance
and autocorrelation might be used interchangeably and the scale diversion would be specified as the
auto correlation coefficient.
Now, these equations are nice, but how will they help us choose?
Q Now that we understand what the autocorrelation is, we can discuss the auto correlation function.
Suppose that we have some time series, why one up to why Big T if we take the correlation between each
point in the Time series and each point in the Time series, we will get a big matrix of size that big
T by Big T.
In fact, you've already seen how we can do this, but actually this won't even work because we don't
even have multiple samples of the Time series.
We are only given a single time series.
That is, we only have one version of one, one version of Y two and so on.
In order to take the sample autocorrelation or auto covariance, we would need multiple Y ones and multiple
Y twos.
So we can't use the typical formula for the sample autocorrelation that you saw before.
In fact, it turns out that the auto correlation function is defined as if the time series were stationary,
as you recall, stationary either weeks and stationary or strong sense stationary implies that the auto
correlation remains constant over time.
Since that is the case, the auto correlation is only a function of the distance between any two data
points.
Why it's one and why T two will denote this distance tau.
So T one minus two is equal to tau.
Therefore the auto correlation is a function of tau.
And since we assume that the auto correlation remains constant over time, we can use all of the samples
from a single time series to compute the auto correlation for any tau.
I understand that describing this in words can be confusing.
However, the formula helps to provide lots of insight.
The components of this formula should all seem familiar.
First, we divide by T minus tau because that's how many samples we are adding up.
We divide by Sigma squared, since without this we would just have the auto covariance formula.
And the auto correlation is the auto covariance divided by the Sigma's of the two random variables inconsideration.
But again, since we are assuming stationary, that means that every point in the Time series has the
same sigma.
And so that's why we divide by Sigma twice.
Finally, inside the summation we have y a t minus mu times y a T plus tau minus mu, which is exactly
what goes inside the expected value for the auto correlation.
In other words, after considering every component of this equation, you should be convinced that this
does in fact compute the sample autocorrelation.
So what is the consequence of having an auto correlation function, which is only a function of the
lag tau note that we call this the lag because it's as if we are comparing two of the same time series,
except that one of the time series is lagging behind by the duration.
Well, what we get is a graph.
This graph plots row the auto correlation on the vertical axis and how the leg on the horizontal axis,
usually such a plot is called the AKF and in some languages such as R, this function is built right
in in Python.
This function is included in statistical libraries such as Sipi and stat's models.
Generally you will see a plot like the following where the akef like zero is equal to one, and then
all the rest of the values are smaller than one.
The reason why the HCF at like zero is one is because this is just the variance divided by the variance
or sigma squared over sigma squared, which must be one.
If you want, you can check the equation for the sample autocorrelation to confirm this.
The interesting part of the fly is that it also provides confidence intervals, these confidence intervals
can be thought of as a kind of threshold.
If we see any lagged auto correlations that are greater than the threshold, we would reject that they
are equal to zero and hence say that they are non-zero to state it more casually.
Any point we find in the Akef plot that goes outside of the confidence interval, we can consider it
to be non-zero.
Now it's important to also remember the frequent test interpretation of confidence intervals, since
this is a 95 percent confidence interval.
That means five percent of the time one of the values in the Akef plot will just pop out of the confidence
interval randomly.
Therefore, if you have, say, 40 lag's, you might expect two of them to just be randomly outside
the confidence interval.
For example, you might see one at like 25 and one to like 39.
Usually you can use your intuition to determine that this is happening by chance and is not actually
significant.
OK, so now that we understand what the Akef plot is and how to look at one, how does this help us
determine the order cue for the moving average process?
Well, it turns out to have a very simple interpretation, a moving average process of order.
Q Can be shown to have non-zero autocorrelation up to Lagu.
That is, if I look at an active plot and I see that at lag to the HCF goes outside of the confidence
interval, but after that it does not, then I would choose Q equals two.
Usually it will be the case that the plot goes outside the confidence interval for smaller legs as well.
So as another example, if I look at an active plot and I see that at like five, the plot goes outside
of the confidence interval but stays inside for values greater than five, then I would choose Q equals
five and the same logic applies for a Q equals 3Q equals four or any other Q.
Now, you might be wondering why does it work this way and how do we know that we can do this?
So I'm not going to derive it right at this moment, but I may in the future for some in-depth lectures.
Well, what you can do is you can actually calculate, given a moving average process, what the theoretical
autocorrelation would be.
So recall that in May one process looks like this.
It's as y a time T is equal to a constant plus theta one, a coefficient times the previous error term
epsilon T minus one plus the current error term Epsilon T.
In this case we can calculate row one to be theta one divided by one plus theta one squared row to row
three and so on are all zero.
If we have an inmate to process, we can do the same thing, we find some expression for row one and
row two and we find that row three, four and so on are all zero.
And again, these are just statistics calculations.
Therefore, it is theoretically justified, therefore, and make you process all of the legs up to queue
will have a non-zero autocorrelation.
And that is what we are looking for when we look at an akef PLI.