Now that we've been talking about multiple random variable is happening at the same time, we can generalize
the single random variable into a vector form.
So now we have a random vector and a basically a random vector is just a vector where each element inside
the vector is its own random variable.
So, for example, here we have a random vector X and we have random X1 x2 all the way up to X and all.
We have this random vector here Y and that has random variable is Y one y two all the way down to Y.
And so this is just a mathematical way of writing groups of random variables together in vector form.
Now, the main of the random vectors can be calculated in the same way as a single random variable,
but it's done on per element basis.
So if we have a random vector X here, the main vector or the expected value of this random vector X
is simply just the expected value of each of the different random variables individually.
We can also work out the covariance so the covariance between two random variable vectors can be calculated,
which forms a covariance matrix.
So instead of a single value is now a matrix.
So let's say we have a random variable vector X and Y where we have the random variables X one all the
way to the end and Y one all the way to Y, and we can work out the covariance matrix using the expected
value of our X random variable vector minus the X main multiplied by Y random variable minus Y mean
transpose.
And again we can simplify this down into this equation here.
So when we do this operation here, we end up with a covariance matrix.
So each element inside this covariance matrix is a covariance value for each of us ran a very, very
each of those random variables with respect to a different random variable.
So this term here is a covariance or random variable, X one with Y one.
Here is a variance on X one with Y two, likewise over across four X one all the way up to M and down.
The Matrix is similar all the way down to extend and Y m.
Since each element in the random variable Vector X itself is a random variable, it is possible to calculate
the covariance of each element in the vector with respect to each other element in the vector.
And when we do this, we get the autocorrelation matrix.
So this is just the covariance matrix for the random vector and itself.
So here we are working out the expected value of our X random variable vector minus its main multiplied
by the same thing transpose.
And this gives us the covariance matrix of our random variable vector X.
And if you have a look at what's contained inside this covariance matrix, it becomes a variance and
cross covariance, which described the probability density function.
So on the diagonal here we have the variances.
So Sigma squared for X1 all the way down to our sigma squared X.
And so these are the variances for the different random variables and our of diagonal terms.
Here are the code, the crosscourt variances.
So these turn to these talk about how correlated the different variables are with respect to each other.
So here we have the cross covariance between random variable X1 and random variable x2.
And down here you can see that we have a mirror image.
We have the cross covariance of random variable x2 with respect to x1 the of diagonal terms, you are
going to be symmetrical and are going to be replicated.
Now if we look at the properties of the covariance matrix, all the autocorrelation matrix, the covariance
matrix is symmetrical.
So this is basically because our correlation or across the variances for IJA is the same as the variance
for Ajai because it's the same random variables.
This means that overall our covariance matrix, our C X is equal to, say, X transpose, because if
we transpose this matrix, the diagonal matrix stays the same and the terms flip around.
It become the mirror image of each other.
But we know that this time here is equal to this time here.
Likewise, this time here is equal to this time here.
So we flip around the diagonal, we end up with the same matrix.
And we also know that our covariance matrixes square because it is and by n matrix.
Now, for a convergence matrix to be valid, it also has to be positive, semi definite.
So this just means that if we multiply our variance matrix by any vectors that here transpose times
by Z, the resulting operation is always going to be greater than or equal to zero.
And this means the matrix is positive.
So this is important later on.
So any probability distribution that has a covariance magic that's positive, some definite is valid.
If it's not positive, something definite and it's not symmetrical, it's not a valid distribution for
probability density function.