In the previous video, we looked at how to calculate the least squares solution to estimating a constant
vector.
We did not use any information about the noise, apart from the fact that the expected value zero or
it had zero mean.
But what happens?
We do know some information about the noise, what happens.
We know how accurate each measurement is or how some measurements might be more accurate than others.
How can we include this information into the solution?
How can we use this information to calculate a better solution than if we didn't have this information?
So now let's look at the simple example that we used last time, but let's extend it.
So again, we have an engine and we have a temperature sensor attached to it.
We're going to look at the relationship between the temperature and the RPM.
So, again, from the previous video, we know that it's going to be a linear relationship that looks
like this.
So as the RPM increases, the temperature increases and it's going to be a function of the two families
x1 x2.
So we're going to take a series of measurements at different CPMs and we're going to have the measurement
y representing the temperature at 8pm r.i so we can have a series of measurements that follow this relationship
here.
And of course each of the measurements is not going to be perfect.
We're going to have some noise on each of the components.
So this is the V.I. again.
So we're going to estimate the premise of this line of best fit X1 next to from all this data here.
So this is what we did on the previous video.
But now we're going to include some extra information.
What happens for if for each of these measurements, we actually have a bit more information?
How about what happens if we knew how accurate each measurement might be?
So what happens if we know that each measurement might have a certain variance of a sigma squared?
So you can see here that each measurement might have a different uncertainty.
So using this information, how can we use this information to come up with a better estimate of what
the line of best fit would be?
The solution to the Matrix would be.
So let's put this into mathematical terms, we have the Matrix equation shown here that we want to solve.
So why eyes are going to be the estimates matrix is going to be the model matrix, the linear matrix
that describes how the measurements are a function of the estimates.
So X are going to be the estimates and V.I. It's going to be the noise.
So this is the same problem as we had last time, but we're going to include a bit more information
in the problem definition.
First off, again, we're going to assume that noise has your name is going to be uncorrelated.
So this is just saying that for each of these values here, the size of one does not affect the other.
There's no correlation.
So if the one is large, is no reason to think that the one of two is small or large or anything that
has to do with the noise up here.
So each of these measurements can be considered independent.
But now we're also going to include a bit more information.
We know that each of the measurements is going to have a certain accuracy.
So we know that the variance so V squared the expected value of that is going to equal our sigma squared.
So this is going to be how much uncertainty we have on each of the measurements.
We won't know what the measurement value is exactly, but we know how accurate it might be.
So using this information here, we can form a covariance matrix so that we've gone through that in
the past.
We're going to define the covariance matrix, are adjusted covariance of the different measurements.
And all that means is just we're going to have a diagonal matrix here and each of the diagonal elements
is going to be the variance of that single noise measurement.
And you can say there's going to be zeros everywhere else because each of these measurements are uncorrelated.
So it's just a single diagonal matrix.
So the problem equation that we want to solve again is this matrix equation here.
So we have the measurements, we have the moral matrix, we have the estimate and we have the noise.
We also know that the variance on each of the noise measurements is going to be equal to sigma squared.
So each measurement can have a different variance or different uncertainty.
And we know that the covariance matrix of all the noise or the uncertainties is just going to be equal
to this matrix and it's just going to be a diagonal matrix with zeros on the other elements because
there's no correlation between the different noise.
Variables are all independent.
Now, we know that to get a good estimate of X, we want to minimize the error residuals.
So that's the difference between the measurements and what the predicted measurement would be if we
had a good estimate.
So this is the estimate.
This is the measurement variance for each of the different measurements independently.
And we can put them into a matrix equation again to work out the measurement residual.
Now we're going to change the cost function from last time.
So the cost function last time just minimize the sum of the residuals squared.
But this time we're going to wait eight with each residual by the inverse of the measurement uncertainty.
So I can see that we have these segments here.
So this is basically saying the more uncertainty we have of each of the measurements, the less effect
it's going to have.
So if we knew this one very, very precisely, this time here is going to be weighted more heavily than
this term over here if this was a lot larger.
So basically saying the more certain we know, the more effect it's going to have on the cost function.
So this least squares system here can be written out in matrix form again of the error residual matrix
transpose times the inverse of the matrix.
So the inverse of the uncertainty times the matrix measurement residual again.
So this is very similar to last time, is that now we have this inverse matrix here.
So again, to work out what our best estimate is going to be, it's just going to calculate by minimizing
this cost function here.
So if we minimize this cost function here with a X, what it means is going to make this X hat as close
to being to the real value of X as possible.
So now, like last time, let's expand the cost function.
So this is the cost function and now we know that we can fill in our different equations for the Matrix
residuals.
So if we go ahead and do that, we fill in a Y minus X and again, so we can expand this whole out,
so we can multiply each of these elements together, expand it out, and we end up with this long equation
here.
So this is going to look slightly different than they squares equation cost function, because now we
have these inverse R functions inside the equation.
So again, to find the minimum, the first step again that we want to do is we want to differentiate
this cost function with respect to our estimate X.
So if we do that, we end up with this equation here.
So this is the derivative of the cost function with respect to the estimates.
So you can see that this term here comes from these two terms here.
This last term here comes from this last time here.
So it's going to be the derivative of the cost function.
And we know that this is going to be a quadratic surface.
So there's only going to be one minimum point and that's going to be the stationary point.
So define the stationary point.
We set the derivative to equal to zero and then we solve it for X.
When we do that, we end up with this equation here.
So this looks very similar as last time, except now that we have our investors are here and invest
here.
So this equation is the least squares solution for when we have a weighted estimation.
So when we have some measurement of uncertainty on each of the measurements of LLI.
So now that we know some information about the uncertainty of the measurements, it would be good to
now calculate how that uncertainty affects the uncertainty of the estimates.
It'll be good to know how certain we are of the estimates so we know how much error there is in the
measurements.
We should be able to estimate how much error is in the estimate from from those measurements.
So if you look at the way at least squares solution, we have a solution that looks like this.
So the matrix here is this combined time here that we just calculated on the last slide.
So if you multiply our errors, so if you multiply our measurements by this, a matrix, which is this
matrix here we get the estimate of X.
Now, if we look at this form here, we want to work out how errors in Y affect errors in X, and we
can do this by looking at the principle of transformation of uncertainty.
So in general, if we have an X Factor, a matrix and another effecta, if we know the uncertainty in
X, we can transform it to the uncertainty in F by using the model Matrix A and we use this relationship
here.
So the covariance of X or the uncertainty of X, multiply it by our AI and then a transpose is going
to be our uncertainty or covariance matrix in F so we can apply this process here to this matrix solution
over here.
So we know the uncertainty in our Y measurements is going to be our metrics.
We can just fill in this equation so we end up with the R, the uncertainty in Y multiplied by a matrix,
which is this transformation matrix here.
We're going to end up with the uncertainty index and we're going to call that uncertainty and the covariance
of X as P.
So pay is a covariance matrix of the uncertainty.
So if we fill in these equations here and fill it in, fill in, are we actually come up with a more
simplified form?
So we end up knowing that the uncertainty in X or how good the estimation, how good the estimate of
X is, is actually just a function of the inverse of the variances of the measurements multiplied by
our matrix of the model and take the inverse of that.
So this is how we can work out how accurate our solution is or how good our estimate is based on our
understanding of the noise in the system or the noise of the measurements.