In this video, we will look at how to estimate a constant picked from a series of noisy measurements
that relate to that constant.
For example, we might have an engine and we might have a temperature sensor attached to the engine.
Now we want to estimate the temperature of the engine using the sensor measurements.
However, the quality of the sensor might not be all that good at each temperature.
Measurement might be very noisy.
So the true temperature of the engine might look like this, but each of the measurements that we get
from the sensor might be very noisy.
It might be scattered like this.
Every time we would query the sensor, we're going to get a different measurement.
And because of this, we want to take multiple measurements with the sensor to try to get a better estimate
of what the true temperature really is.
So let's put this into mathematical terms, suppose that we have key measurements where each measurement
is denoted by the temperature measurements in this case are going to be a function of the quantity X,
which is the true temperature, and they are corrupted by some noise V.I..
Now we will assume that V.I. is random noise and this noise has zero mean and is uncorrelated white
noise.
This just means that the expected value for the random variable, which is the noise, is going to be
equal to zero.
Now, to estimate the true temperature from all the measurements, a reasonable idea would be just to
average all the temperature measurements together that have been made.
So if we sum up all the measurements, we're going to have an equation that looks like this.
So the main measurement denoted by Y Bar is just can be one of acoustic measurements summing up all
the different Y eyes.
So we substitute in our measurements into this equation here.
We're going to end up with this.
Now we can see here that X here is going to be a constant it doesn't change with the each of the summation.
So we can actually bring out the front.
So we can see here that the main of all this, since the measurements, is going to be equal to the
true temperature X plus the summation of all the random noise.
Now, we know from probability that this operator here looks very familiar.
This is actually just the expectation of radar.
And we know that the expectation of right of a random variable has zero.
I mean, is just going to be equal to zero.
So this is why we can say that the best estimate of the true temperature, so the best estimate being
set here is just going to be who the main of all the measurements this is.
Because if we have enough measurements, the expected value of all the noise values is going to equal
to zero zero main and will be left just with the true temperature.
This relies on the fact that if we take enough measurements, the average value of all the noise should
be zero nine.
So it should all cancel out.
The number of measurements that need to be made will depend on the required accuracy of the estimates
and the size of the noise distribution from the sensor.
Let's extend the estimation of a constant skalla in the previous example to a constant factor.
So suppose that we have the temperature sensor again attached to the engine, but this time we're going
to run the engine at different RPM so speeds and we're going to see how the temperature changes.
So when we do this, we get end up with a graph that looks like this.
We can see the temperature increasing as the engine speed increases.
So now, instead of estimating a single temperature, what we want to do is we want to estimate this
relationship here.
So we want to work out how the temperature changes with the RPM of the engine so this relationship can
be written as a linear line equation.
So the temperature y is going to be a function of a long period of X one times higher RPM plus X to
another parameter of the line.
So this is going to be the slope of the line, this valley up here.
This is going to be the bias of the lines of how far it's shifted from the origin.
So using these two parameters here, we can come up with a line of best fit.
So somehow we have to estimate X one next to from this data set here from the set of measurements.
So, again, putting this into mathematical terms, we're going to end up with Kay measurements and
this temperature measurement, why it's going to be a function of the R.P.M. measurement, the noise
and the line parameter for the slope and the line parameter for the bias.
And again, we make the assumption that the noise on the measurement is zero, remain uncorrelated and
it's white noise so we can take all the equations for all the different measurements we've made and
put it into a matrix equation.
So this matrix here, why is going to be a vector of all the different measurements?
He is going to be the measurement matrix, which is going to describe the linear system that we want
to estimate.
X is going to be the thing that we're actually estimating and V is going to be the vector of all the
noise.
So if you put that into a matrix equation is going to look like this.
So this is just a matrix matrix way of writing out all these simultaneous equations.
Now, this is the equation which we would like to solve now, when you knew all the different terms.
Exactly, then maybe just rearrange this equation and solve for X directly.
But the problem in this is that we have this random variable here.
We have this noise vector and we don't know what the noise Victor is, but we do know some properties
of the noise vector.
We know that the expected value of the random variable for noise is going to be equal to zero.
So we can use this property to come up with an estimate.
So instead of solving for X directly, we actually want to solve for estimated X, and we can do this
by looking at the measurement residual.
So if you rearrange this equation here, instead of trying to solve for X, we're going to solve for
our hat.
So this is going to be an estimated value of X.
Now as we make this measurement residual here go towards zero, it basically means that we're going
to minimize all the effects of the noise.
So this equation here is basically just a rearranging of the Matrix equation to work out the different
measurement residuals for each of the different measurements.
So we define a cost function as the sum of the squares of all the different measurement residuals.
So that's this equation here.
And this equation here can be written out in a vector form of just the residual vector transpose time,
the residual vector again.
So this vector here is going to be a scalar and it's going to be based on the size of the measurement
residuals.
So the larger the residuals, the larger the cost function, the smaller the cost function, the smaller
residuals.
So the idea behind these squares estimation is to minimize J such that our estimated value for X goes
towards the real value of X.
So now let's have a look at how we mathematically go about minimizing the cost function.
So first thing I want to do is expand the cost function.
So if you ride out the original equation for the cost function, it is a function of the measurement
residual vector.
So if we substitute in an equation for the residual vector, we end up with this equation here.
And again, we can multiply all these terms out and we end up with the full expansion of the cost function
so we can see that the cost function, our scalar value J is going to be a function of the measurement
vector Y, our estimate X and our estimation matrix.
So what we want to do is you want to find the value of our asset that minimizes this whole equation
here.
So now we want to find the minimum of the cost function and we can do that in a very similar way as
we would find a minimum of any function, first thing we want to do is differentiate the cost function.
So if we take this function here and we differentiate it with respect to X, we come up with this equation
here.
So now from our maps, we know that if we want to sit, if you want to find the minimum, we can set
the derivative to equal to zero to find a stationary point.
And since this equation is a squared equation, we know that there should be only one stationary point,
which is going to be the minimum point.
So we can substitute in zero for this term here and we can rearrange this equation to work out what
Xs.
And if we do that, we end up with this squares equation here.
So this equation is for the least squares solution.
This is the equation that minimizes the sum of the squared error.
For this equation to be tractable, the matrix here must be full rank and the inverse of this matrix
here must exist.
So the number of measurements K must be greater than the number of elements in the X Factor that we
want to estimate.
So let's have a look at a concrete example, so we've taken a number of sensor measurements of the temperature
at different RPM's and we have a table here.
So these are the temperature measurements.
These are the R.P.M. measurements.
So we can form a Y vector, which is just going to be the elements of the temperature sensor measurements.
And we can form our matrix.
And we know that the linear relationship that we are trying to estimate is going to be the odd P.M.
and a one so that when we multiply this equation out here, we get the equation of a line.
So we end up with this matrix here where we have the RPM measurements one, two, three, four and five.
And then we have the constant BI's elements, just one one, one one, so that when we multiply this
matrix out here, we're going to get X1 times A1 plus X2 and same thing here we're going to get X1 times
are two plus x2.
But once we have these two equations here, we can go about calculating at least squares function.
So first thing we want to do is begin to use the matrix and multiply the transpose times.
If we use a matrix on the previous slide, we're going to come up with this equation here.
Now, the next step in a solution is to work out the inverse of this equation.
So the inverse is going to be this equation here.
The next step is to multiply the inverse by the makeshift again.
So we get this solution.
And then the last step in the least squares equation is to multiply this matrix by the measurement vector
y.
And again, so this is our final solution here.
This is going to be the parameters for X1 and X2.
So if we take these parameters out from this solution, Matrix X, we end up with the linear relationship
that we want to estimate.
We want to estimate the temperature first R.P.M. relationship.
So the temperature for a given RPM is going to be X1, which is our nine point one.
So 91 divided by 10 X1 plus are biased in Obama's term.
Here is fifty two point seven or 527 divided by ten.
So this is the X2 term here.
This is the two term here.
So this equation here is line of best fit that fits the data on the previous slide.