Welcome back to practical time series analysis.
We're looking at stochastic processes and their realizations time series.
In trying to gain traction on them by developing some
properties that'll allow us to get work done.
One of those properties is the concept of weak stationarity.
We've seen that noise is weakly stationary.
We've seen that random walks are not weakly stationary.
And in this lecture,
we try to show in a formal way that moving average processes are weakly stationary.
In this lecture, we'll look at the ACF.
Here we're looking at the autocovariance function of a moving average process and
then we could get the autocorrelation function.
Start with some building blocks, some IID random variables.
We'll start with Z of t as our building blocks, iid with mean 0,
that'll be important to us and constant variance, sigma squared.
Recall that we build moving average process of order q by
starting at Z sub t and looking back in time to Z of t- q.
And, just taking a sum with some weightings.
The weightings are given by the betas.
In order to develop our result, we'll remind you of a basic idea on covariance.
We saw this in elementary statistics, by looking at summations and
coming up with the so-called computing formulas.
Here, I'm thinking about random variables and using operator notation.
The covariance of two random variables is going to look like the expected value of
the product minus the product of the expected values.
So immediately, perhaps trivially, if your random variables have zero mean,
you can get their covariance just by looking at
the expected value of the product, that'll really help us moving forward.
If we're going get the covariance of random variables Xt and
Xt + k then we'll be able to use this idea to get a very nice result.
Let's not make more of this than we need to.
X of t is built on a bunch of independent identically distributed Z sub t's.
Same think with the Xt + k.
K is just telling you how far away from each other random variables X of t and
X of tk are along our stochastic process.
If k is big, bigger than Q, then the random variables Xt and
Xt + k are being built from different independent underlying random variables.
They're not being built by anybody in common, any of the random variables in
common and so we wouldn't expect them to have any relationship to each other.
They're each just being built from different sets of noise variables.
Working a little bit more rigorously, the expected value of each of these is 0.
So we can obtain the covariance just by looking at the expected value of
the product.
There are more elegant perhaps,
more sophisticated ways to do since our mathematical preliminaries in our course
are trying to be kept as sort of basic as possible, though we expect many people on
the course have very sophisticated mathematical backgrounds.
But, just to make sure we keep everything as basic as possible,
we'll look at the covariance of two random variables along our moving average
process as the expected value of the product that you see in these brackets.
Again, if the underlying Z of t are independent,
we won't get contributions to the covariance unless X sub t and
Xt + k are close enough together.
If we expand the product, this looks a little bit tedious but
we can just work it term by term.
If we expand the product by multiplying out in the most simple minded way
possible, we see that to get the covariance, we need the expected value or
this term in a square brackets.
Now the Z sub t's are independent and since the expected value of the product is
the same as the product to the expected values for independent random variables,
when the subscripts disagree, like t and c + k, if k is not equal to zero,
when the subscripts disagree, you don't get any contribution.
This, the term, the individual term will evaluate to zero.
So what we'll look for
in the sum would be terms where the subscripts on the Zs actually do agree.
That's our basic idea here in coming up with the covariance function.
For little bit of intuition, this is how this works.
When k is = to 0, we'll come down to basically just the variance of X sub t.
When k is = to 0, the only place we're going to get contributions to our
sum will be along the main diagonal of the sort of implied, is not exactly matrix but
this sort of implied diagonal here in the matrix in our bracket.
That means that when k = 0, the expected value works out as sigma squared.
And then we'll just take the sum of the individual coefficients squared.
When k is = to 1, again, we're going to look for
pairs of Zs in our brackets where the subscripts agree.
When k is = to 1, it isn't along the main diagonal where we get agreement now,
but we look up 1.
So, up 1 in terms of the diagonal and
perhaps you could say to the right on each one of the terms.
So, I'm seeing Z sub t, Z sub t agreeing here.
Down in this row, we will have Zt- 1 Zt- 1 agreeing, continue
all the way down until you wind up at the very last contribution down here.
When you work out what the formula look like,
you will factor out your sigma squared and
come up with the sum i = 0 to q- 1 beta i beta i + 1.
When k is = to 0, we had q terms and now, we have one fewer term.
Actually, we had q + 1 terms that are main diagonal and now we had q terms.
When k is = to q, they'll be fewer terms still that are involved.
Again, we're just looking to see where the subscripts might agree.
And we're only going to catch that up in this, upper right hand corner.
Every other term will evaluate as 0, giving a sigma squared beta 0 beta q.
One last slide and then we're done.
When k is less than or equal to q,
obviously if k is greater than q no contribution.
When k is less than or equal to q,
this chart here is meant to show you how the contributions are going to behave.
I have x sub t in my head up here, I have
x sub t + k and we move through and
we look at the underline Zs and the coefficients involve on those Zs.
And we do this for both of these.
Here's your Zt + k.
Beta nought through beta q.
The only place where you're going to get a contribution is where
the subscripts agree and that's on these terms here.
If you look to see how are we going to build the total term then,
we'll have beta nought and beta k.
Beta 1 beta k + 1 and we'll keep on going
until we get at beta q- k and beta q.
When you form the sum then, sigma squared comes
out from Z sub t, Z sub t, Z sub t -1, etc.
We pulled that out.
And we get a sum on our beta terms.
The important things to note in this very important formula
is that there is no t dependence.
It doesn't matter where along your stochastic process you're looking.
The covariance of X sub t and
X sub t + k do not depend upon location along the random,
along the stochastic process but only on the separation of the random variables.
That's the very idea of weak stationarity.
So, in this lecture, we've shown that moving average processes or
weakly stationary.
And we've also come up with an explicit formula for the covariance of
two different random variable along our moving average process.
In the next lecture, we'll look at another canonical stochastic process or a very,
very important basic stochastic process, the autoregressive process.