Hello everybody,
in this lecture we'll talk about Yule-Walker equations in matrix form.
Our objective is to rewrite the YULE-Walker equations we have already seen
In a matrix form for auto regressive processes of order p.
So let me remind you ARP processes are where the Xt
is regressed on the t previous values starting from t- 1 until t- p.
Right, so that phi determines the order of the process,
phi 0 is a constant that might come there in our process.
And Zt is our innovations, random disturbance, or white noise which
is normal with some sigma standard deviation.
So what we would like to do is to somehow use your equation
in matrix form to estimate this parameters but this comes a little later.
For that reason, we need your equations in matrix form.
Let me just note the following,
if you actually take expectation of this model right here, this P model.
If you take expectation, assuming the model is stationary, so
this is stationary RP process.
Expectation of XT is mu.
Expectation of XP is mu.
Expectation of X minus 2 is mu.
An expectation of xt minus p is mu, expectation of the random
noise is zero from this, you can actually find mu, right?
Mu can be calculated in terms of these coefficients, but that's not what I would
like to do, I would like to subtract these two equations side by side.
For example, if I subtract xt minus mu this phi 0 will go away and then I will
plot view 1 that will plot view 2 and this is basically the expression we obtain.
Realize the following if I define Xt minus mu as a new random variable,
call it Xt/t, it's basically the previous random variable shifted.
To the right or left depending on the sign of mu but
interesting thing is that expectation of tilda t now is actually 0.
In other words, if I replaced this by x tilda t,
then I get this ARP processes without a constant, which would have 0 mean.
So in other words,
we're going to basically work on AR[p] process with mu = 0.
And in examples wherein mu is not [INAUDIBLE] Xt and
we will try to fit the model AR(p) model with expectations here.
Okay, so this is our model, our model is Xt regressed on the p values,
previous p lags, plus some random noise.
Remember Yule-Walker equations.
Yule-Walker equations questions for this process will be 0K equal
to a different equation for 0k minus 1 unto RK minus P and
this is always true for K greater or equal to 1.
And then remember that rho 0 at correlation is always 1 or
the itself right every random available auto correlation with the self is one.
And for the negative values we can basically write one negative K and
calculate negative K.
So let's write this the correct way for
different values of K from one to P, all of them.
So we're going to have P equations here, sSo this is our Yule Walker equation.
I plug k equal to 1, this becomes row 1, this becomes row 0,
this becomes row -1, this becomes row 1- p, right?
-1, -2, 1- p, all these negative values here.
If I plug 2 in, This is rho 1 this is rho 0,
then I will have all seven negative values, rho 3 and so forth.
So we basically rewrite this Yule-Walker equation for every value of k from 1 to p.
So we get p equations but then we remember the following, rho k is rho -k, right?
So rho -1 is rho 1, rho -2 is rho 2,
so every rows with negative indices, row negative 1, row negative 2.
We can replace them with row 1, row 2, and so forth, so let's do that.
So if I put row negative k to row k, for every possible value of k
in that system of equations, we will obtain the following system.
In other words, row negative 1 now is gone, we have row 1 for
when -2 is gone we have row 2 and rho -1 here is going to be we have rho 1.
This one was rho 1- p, now it is rho p- 1, so we get this new system of equations.
And we write it
by realizing that they're all 0 the auto correlation of lack 0 is always 1.
So basically, rho 0 is 1, so this is just phi 1, this is just phi 2,
this is just phi 3.
Basically the diagonal we get rid of our rho 0s and
we get these system of equations and we can write this in a matrix form.
Realized this is the r coefficient matrix, so this is our coefficient matrix,
we'll call that r coefficient matrix and
this is our coefficients of the process, phi 1, phi 2, phi p.
And on the left-hand side, we have rows from 1 to p.
Okay, so in other words, I can write this as b = R f R being A p by p matrix.
Phi is p by 1 matrix, b is p by 1 matrix.
So if I would like to solve this, remember at the end of today,
I would like to get this coefficient.
If I would like to write this,
I Trying to find inverse of R if it exists, it turns out that it exists.
R is a very nice matrix inverse of R exists.
If we multiply inverse from both sides,
we get phi which is the coefficients of my system equal to R inverse times B.
Now when we are actually trying to fit ARP
process to actual time series data set, of course we do not have.
We do not have autocorrelation function,
we actually have sample autocorrelation function.
So how do we update the whole process?
Well, every time you see roles, we replace them with Rs and
we try to find phi's, right?
So instead of role rows here I have r1, r2.
I call this b hat, instead of r matrix, we call this r hat.
This is approximation and
instead of phi, all of these should have hats in here, we get phi hat.
And then we estimate phi hat by finding inverse of not the original r but r hat.
Okay, so let me just mention that these Matrices R and
R Hat--both of these are symmetric matrices.
They are positive semidefinite matrices.
In other words,
all eigenvalues are nonnegative and their inverses actually exist.
In other words, if you look at this equation, b equal to r or
b had equal to r had, had, we can actually solve this, there's a unique solution.
So let me give you an example, so we will estimate of the following model.
Assume that we have this AR2 process with the no mean,
zero mean and we would like to find first r1 and r2.
This is example autocorrelation, we can find them using acf working in R.
And then all we have to do is basically solve the system of equations for
phi 1 hat and phi 2 hat both of these will be an estimation for phi 1 and phi 2.
If we have AR(3) process, basically if we are doing the same thing,
we have three coefficients now.
We're going to have three by three systems, this is our r hat,
this is phi hat, and this our r, b hat matrix and we were trying to solve this.
And this matrix has an inverse, we'll be able to find phi 1, phi 2,
phi 3 hats here, so what have we learned?
We have learned the matrix form of the Yule-Walker equations, which is b = r phi.
And we have learned how to estimate the coefficients
of AR processes using Yule-Walket equations