We have looked at using random variables individually in the past examples, so now we're going to look
at the concept of using multiple random variables in a single experiment.
So let it be a random variable with a PDF of our effects here.
Also, let let's why be a random variable with a PDF of F why here it is now possible to define a PDF
for the joint probability that is a probability of X and Y, and we define this as a probability density
function of F, X, Y or for shorthand notation.
We can just write it as F with X and Y being the parameters of the input.
So we automatically know that this is a probability density function for random variables X and Y,
so we can either use either of these annotations.
So just like in basic probability, the joint probability gives us the probability of X and Y happening
at the same time.
So from the probability density function, the joint one, we can work out the joint probability and
we can just do this by using our probability equation that we had before.
But we're going to extend it to being a function of two variables.
So now we have our ingroup from our limits and B for the X variable and we integrate the function with
respect to X, and then we also integrate the function again with respect to Y between C and D to get
the limits for C and D.
So this joint probability equation here gives us the probability that the random variable X is between
limits A and B, with the joint probability that Y is also between limits C and D.
So this is just an extension of our equation to work out the probability from the density function using
the integral.
But we now we have to do a double integral because we got two parameters that we want to integrate over.
We can also work out our marginal density functions so we have a joint probability density function
for random variables, X and Y, if we integrated over the opposite variable, we can get the probability
density function for the other variable.
Sorry to work out our density function for a random variable X, we have to integrate our joint probability
over Y and to do the opposite.
If you want to work out our density function for the random variable, why we have to do our indefinite
integral over the function with respect to X.
We can also have a look at the expected value for when we're using a joint probability density function,
so we recovered in the past that the expected value for when we're using a single random variable is
just this equation change here.
So so we have a function of X and we want to work at the expected value of it.
The answer this is just simply the integral of that function as a function of X multiplied by the probability
density function of X integrated, all with respect to X..
Now, when we go to our multiple random variables, so now we just have to extend it by using a double
integral over our X and Y.
So it's pretty much the same equation, but now we're using a double integral for both parameters.
So just like in basic probability, we can have independent random variables, and what this means is
that if we have events and B, if they're independent, it means a total probability of event A and
B happening.
Is this the multiplication of the probability that happens multiplied by the probability that Event
B happens?
So if they're independent, this relationship holds up.
Now, a similar relationship can be found for the probability density functions for all random variables.
So if we have a joint density function of X and Y, if events, all the random variables, X and Y are
independent, it means that the density function is just going to be the same as multiplying the two
individual marginal density functions together.
So if this definition holds, it means that the random variables X and Y are independent of each other.
So now let's have a look at the expected value for multiplication of independent random variables.
So we want to work out the expected value when we multiply a random variables, X and Y together.
So now using the equation that we showed on previous slide, we can work out our integral.
So we just multiplying our random variables X and Y, so that becomes a function and we multiply that
by our density function.
So we can expand this equation out further.
We actually know that our joint probability density function is independent, so we know that we can
split it up into our F and Y random.
So we know that we can split it up into our X and Y probability density functions.
So we end up with this equation shown here.
So we're basically just splitting up our joint density function into the two individual component density
functions.
So next we can move and separate out X and Y variables together so we can separate them out.
And now we have two integrals multiplied together.
So we have all the X terms in this integral and all Y terms in this integral area here.
Now, if you have a look closer at these two integrals, you might look at them and they might seem
pretty familiar.
This is because this term here and this time here are simply just the expected values for the X and
Y random variables.
So using this here, we know that our definition, assuming independent expected value for random variables
X multiplied, is just going to be the expected value of the X random variable multiplied by the expected
value of the Y random variable.
And of course, this only holds if X and Y are independent.
We can also have a look at the expected value of the sum of independent random variables, so a function
Z here is a function of two parameters, X and Y, and this function is just the sum of two independent
functions, a function G, which is just a function of X and function F and and function Hech, which
is a function of Y.
So we want to work out the expected value of our function Z and this just turns into our expected value
of our function.
G.
Function H.
So now riding out our expected value equation, we can put in a function and put the function that we
want to work at the expected value of in front, you multiply it by the probability density function.
Now, we can expand this sum out here into two terms.
We can also expand out our probability density function.
So if we assume X and Y are independent, we know that we can break it down into two different individual
probability density functions.
So when we do that, we can get this equation here.
So we splitting out our function of G into this one.
A function of how much into this one over here.
Multiply it by our two individual probability density functions.
And like we did in the last example, now we're going to split this in this double integral into an
integral respect to X, an integral in respect to Y, because this is all just a single multiplication.
So when we do that, we end up with this term here and we can also have a look at a different individual
integrals and we might notice some commonalities between the two.
Now, if you look at this integral here, this integral here is just going to equal to one, because
this is just summing up the probability density function and we know that the probability density function
has to equal one.
We can do the same thing over here for X, that also has to equal one.
So now we just end up with the sum of this integral and this integral and going back to our basic expectation
in the integral, we can see that this integral here and this integral here are just the expected values
for a function G and a function H.
So this is basically saying that when we say this is saying that our expected value for the sum of two
independent random variables is just the sum of the expected values for the two independent variables.
So if we had our function, Z is equal to X plus, why then our expected value for this function is
just going to be expected value for X plus expected value for Y.
And again, this only holds when X and Y are independent.
So these are two different identities for independent random variables that are useful to know.
So now we can also look at dependent random variables, so the two random variables might not be independent,
there might be some correlation between the two.
So every time we query the joint custody function, we might end up with some correlated examples.
So we can see on the right here, we have a graph that has the X random variable and Y random variable.
And you can see that every time we create the density function, we might get different points that
look like this.
And it's pretty clear to see that these are correlated in some regard.
We can see that there's basically the following in this case, a line of correlation.
And this is actually a positive correlation because this means that as X gets bigger, Y gets bigger,
you could have cases of negative correlation.
So that would be on the other side when when it gets bigger, Y gets smaller.
So it'll be along the opposite slope or you might have zero correlation.
So there might be independent and that would just mean that random all over the place.
There's no pattern to them.
So to work out the correlation between the different components, we can look at the covariance.
So in the past we looked at the variance of a parameter and that was just the expected value of the
random variable, minus the main squared.
So the covariance is very similar, but instead of being a function of a single random variable is now
a function of two random variables.
So here we have the expected value of the multiplication of a random variable X minus its main multiplied
by a random variable, Y minus this mean.
And we can expand this out and we can end up with this equation here.
So so this is how we can work out the covariance coefficient.
So we basically look at the expected value of X and Y minus the mean of X and Y.
So using the covariance we can work out a correlation coefficient.
So that's basically this term here.
So using the variance, multiply that by the standard deviation for the random variable, X and Y gives
us a correlation coefficient between negative one and one.
Negative one means that negatively correlated positive one mean positively correlated zero means are
not correlated at all.
So here is a summary of all the different concepts we have gone over in this video.
We've talked about the joint probability density function and how to work out the probability from it.
We've described how to work out the expected value equation for when we have a joint probability density
function.
We looked at the independence condition for a joint density function, just being the sum of the two
individual density functions.
We've looked at the concept of covariance.
So how much?
One parameter is correlated with another parameter.
We've worked out how to work out a marginal density functions from our joint density function here and
we've gone.
A few independent expectation identities will come in handy every now and again of that.
The expected value of the sum that the expected value of the multiplication of random variables is just
going to be expected.
Value multiplied together and likewise with the sum is just going to be the sum of the expected values
together.
Now that we've been talking about multiple random variables happening at the same time, we can generalize
the single random variable into a vector form.
So now we have a random vector and a basically a random vector is just a vector where each element inside
the vector is its own random variable.
So, for example, here we have a random vector X and we have random X1 x2 all the way up to X and all.
We have this random vector here Y and that has random variable is Y one y two all the way down to Y.
And so this is just a mathematical way of writing groups of random variables together in vector form.
Now, the main of the random vectors can be calculated in the same way as a single random variable,
but it's done on per element basis.
So if we have a random vector X here, the main vector or the expected value of this random vector X
is simply just the expected value of each of the different random variables individually.
We can also work out the covariance so the covariance between two random variable vectors can be calculated,
which forms a covariance matrix.
So instead of a single value is now a matrix.
So let's say we have a random variable vector X and Y where we have the random variables x1 all the
way to end and Y one all the way to Y, and we can work out the covariance matrix using the expected
value of our X random variable vector minus the X main multiplied by Y random variable minus Y mean
transpose.
And again, we can simplify this down into this equation here.
So when we do this operation here, we end up with a covariance matrix.
So each element inside this covariance matrix is a covariance value for each of those random very,
very each of those random variables with respect to a different random variable.
So this term here is a covariance or random variable, X one with Y one.
Here is a variance of X1 with Y two.
Likewise over across four X one all the way up to I m and down the matrix is similar all the way down
to extend and Y.
Since each element in the random variable Vector X itself is a random variable, it is possible to calculate
the covariance of each element in the vector with respect to each other element in the vector.
And when we do this, we get the autocorrelation matrix.
So this is just the covariance matrix for the random vector and itself.
So here we are working out the expected value of our X random variable vector minus its main multiplied
by the same thing transpose.
And this gives us the covariance matrix of our random variable vector X.
And if you have a look at what's contained inside this covariance matrix, it becomes a variance and
cross covariance, which described the probability density function.
So on the diagonal here we have the variances.
So Sigma squared for X1 all the way down to our sigma squared X.
And so these are the variances for the different random variables and our of diagonal turns.
Here are the code, the crosscourt variances.
So these turn to these talk about how correlated the different variables are with respect to each other.
So here we have the cross covariance between random variable X1 and random variable x2.
And down here you can see that we have a mirror image.
We have the cross covariance of random variable x2 with respect to x1 the of diagonal terms, you are
going to be symmetrical and going to be replicated.
Now if we look at the properties of the covariance matrix, all the autocorrelation matrix, the covariance
matrix is symmetrical.
So this is basically because our correlation or across the variances for IJA is the same as the variance
for Jaci because it's the same random variables.
This means that overall our covariance matrix, our see X is equal to, say, X transpose, because
if we transpose this matrix, the diagonal matrix stays the same and the terms flip around.
It become the mirror image of each other.
But we know that this time here is equal to this time here.
Likewise, this time here is equal to this time here.
So we flip around the diagonal, we end up with the same matrix.
And we also know that our coverage matrix is square because it is and by N Matrix.
Now, for a convergence matrix to be valid, it always has to be positive, semi definite.
So this just means that if we multiply our variance matrix by any vectors that he transpose times by
Z, the resulting operation is always going to be greater than or equal to zero.
And this this means that matrix is positive, definite.
So this is important later on.
So any probability distribution that has a covariance magic that's positive, essentially definite is
valid.
If it's not positive, something definite and it's not symmetrical, it's not a valid distribution for
probability density function.