Welcome back to practical time series analysis.
We've been looking at simple stochastic processes that can be used to generate
the kinds of data that we see in science, business, engineering.
We've looked at moving average processes and in the last lecture or
two we're looking at autoregressive processes.
These are processes where the state of the system depends upon some sort of noise, or
innovation, or shock, together with some recent history of the system.
In this lecture, we'll work on expressing an autoregressive process of order p.
As a corresponding infinite order moving average process.
That sounds like we are about to make things more complicated.
But it's actually going to work for us beautifully as a simplification.
We'll use this to find the ACF fo an auto regressive process.
And we'll drill down and the order aggressive process of order 1.
And look at how the autocovariance structure,
the autocorrelation structure depends upon phi.
A little reminder, we take a series of white noise running variables Z sub c.
We'll take the mean of zero and
sigma squared is how we'll denote the common variance.
The auto regressive process looks like Z of T
time T plus the linear combination of states going back several lags.
The Backshift Operator we've seen is a way to express given a current
position a longer are so I guess suppresses look back by a position.
So B times X sub t is going to give you the system t to times t-1.
If we apply B squared to Xt would that the system t, t-2 etc.
So if we're going to form an expression for the auto regressive process,
we can say x of t looks like noise plus five one times shift on x of t.
There's your x of t minus one all the way down and we'll do it for several states.
How's this going to help us?
Certainly we can start writing things more compactly.
A polynomial that we'll encounter often capital fi of b
is just how we would write the noise if we want to express noise as an operator
on x t and this actually is going to be useful for us.
We could write Xt as 1 / 1- this polynomial term right here.
And then we make the important point that we're going to try to express that
as an infinite order moving average.
So we'll have the time delays and suitable coefficient operating on Zt.
If that's a little too obstruct, let's take a p equals 1 first order example.
Current state at time t is going to look like
some noise plus pi times the state at time t- 1.
But we can just apply this to t-1, this theta times t-1.
So Xt is Z sub t +, now we'll take phi and
we're going to expand t sub t-1, the state of the system one period ago.
As noise one period ago.
Plus phi times C to the system two periods ago.
Nothing's stopping us we're doing the same process for x sub t minus two.
So we'll get to see that the system looks like noise plus phi times noise one
period ago plus phi squared times noise two periods ago.
And, of course, we're going to have to accommodate the state of the system
three periods ago, when we do our expansion.
I've only gone down for a few steps but I think you can see the pattern.
This might be a little easier to see if we use the operative notation
when we do the operator approach.
We'll take system state at time t looks like noise plus phi
times the state one time period ago.
And now we're going to treat our operator almost like it was a number, and
we're just going to do algebra on it.
X sub t is 1 over 1- this term here, times Z sub t.
And let's expand to that around the way we would through geometric series.
I've got 1 over 1- pi times B.
And I'll take that as pi B + pi squared B squared, etc.
And of course, this is the formula that we'd be use.
1 over 1 minus a is the infinite sum 1 and
then we'll keep raising our number to great and greater powers.
We're treating 5 times b as the number a here.
So why did we do all this?
Because now our results are, one would say almost trivial.
The expected value of an order aggressive process of order P will
look at the expected value of state of the system times t.
That's the expected value of these sum here of operators and
proficients are reading on Z's of t.
The expect value of course will distribute over the sum will pull our
coefficient through the expected value operator and will also take the B's.
And we'll apply them to the Z sub t to the noise to get B time Z
sub t is noise period to go B squared Z sub t is noise two periods ago etc.
We are left with the expected value of X sub t looks like.
The expected value of the noise at the current time.
Constant time's expected value the noise one period ago etcetera.
That's an infinite sum.
But all these expected value terms evaluate to 0.
So the expected value of our auto regressive process in fact is 0.
The variances the same approach basically the same.
Manipulation except when we take the variance and we apply it through our sum,
we'll have the variance operated distribute over terms in our sum.
That's true and I've taken liberty of applying the time shift operator as well.
But constants come through the variance operator as constants squared.
So theta 1 will come through as theta 1 squared, theta 2 is theta 2 squared, etc.
The variances are constant, I'll call it sigma sub z here,
squared Just to remind us that this is variance of the noise random variable.
And each of these will produce a sigma squared term, so
I'm going to pull those out.
And we're left with sigma squared times this sum.
Evidently we have the necessarily condition for stationarity.
We would like that variance to be constant, and so
we need that infinite sum to converge.
Moving to the autocovariance, and
we'll get to the autocorrelation in just a moment.
We can take the autocovariance looking as that constant
variance and then we have a sum of terms like this.
Remember we're only going to get contributions where the variables overlap.
Four and A are a P process then, we'll get sigma of K looking like constant
term we're taking the order Q up to infinity so the only thing that
really changes moving from here to here Is the infinite limit up top here.
It turns out that this gives us a very simple formula for the autocorrelation.
We're going to get a fair amount of cancelling and
we'll have terms that look like this.
Let's see some examples.
If we have an AR(1) I have phi here as a generic coefficient.
When we apply our formulas, then the theta i becomes 5 raised to the ith power.
Theta i + k becomes phi raised to the i + kth power.
Our sum is on i.
That's the variable at the site of our sum.
The K is constant, so I'm going to pull pi raised to the Kth power out of the sum.
You can see that right here.
Leaving us inside of the sum with pi raised to the I times pi raised to the I.
So we have a simple formula for the autocode variance
We have an even simpler formula for the auto correlation.
Its just phi raise to the kth power.
Now if we have our coefficients looking like 0.9 and
0.2 we can see that when our coefficient is 0.9 I've called it out for
this graph, this is the phi for the proceeding graph.
We'll have something that to case rather slowly.
You can imagine as the coefficient approaches one,
that the decay will become slower and slower.
If we have alpha point two, that's rather a lone number.
The correlations are going to dot off really quickly.
What happens if we have negative correlations?
If our coefficient is negative as we raise it to succeeding powers, we'll see
that our auto-correlation function becomes negative, positive, negative, positive.
Here the correlate or the coefficient is point 3.
So you get rather rapid decay.
When the coefficient is -.8,
the alternating character looks very pronounced.
And the decay is slower.
What we've learned in this lecture is that an AR(p) process
can be expressed as an infinite order moving average process.
This has the advantage of making some theoretical results very easy to show.
We're able to find the ACF Of a what a regressive processes way.
And we've seen how the coefficient phi is going to
determine the ACF for first order auto regressive process.