We will now take a closer look at the performance and training of the common filter.
How can we tell if the filter is working or tuned correctly and there are two major ways of doing this,
the first way is to compare the comment through the state estimates with the true states or estimates
of what the true state are in the exercise.
So far, we've been using a simulation to attain the truth for comparison and comparing that to the
common field of performance.
But just because we have a simulation that does not necessarily mean that the filter will work in real
life, there might be some differences between the simulation and the actual real life physics and dynamics.
So another way we can do is to use an external reference system.
So on the true system, in real life, we can have another way of estimating what the state is and we
can compare those estimates with the common field of state estimates.
So basically, what we want to do is we want to look at the state estimation error.
So basically, if we know the true state, we can come from the simulation or some external reference
system and we compare it to the common field of state estimates.
We want the state estimation error to be as small as possible.
So we want to minimize this or make this is close to zero.
We also want to make sure that the state estimated covariance matrix for all the time steps is also
representative of how good the estimate is.
So we want to make sure that this relationship is true as well.
Now, in some cases, we might not have a good simulation that we can use to generate the truth, or
we might not have an external reference system that we can put on in real life to generate what the
true signal should be.
So in these cases, we have to use the innovation and we can compare the innovation, statistical properties
to work out how well the filter is performing and the main things that we're going to look at with the
innovation, we want to make sure that it has zero mean and that the innovation sequence variants should
be a similar order of magnitude to the predicted innovation covariance.
So we can see on the slide here that we have an example simulation run and we have the innovation sequences
for this for the X and Y position measurements, when we say we have zero, I mean, we want to make
sure that the police lines here, if we send them all up and take the main we want it to be as they
remain.
And the same thing for the Y.
So if the innovation is centered around zero, it means on average, the main error between the predicted
measurement and the actual measurement is zero.
And this is one of the criteria we assume is true for when we're running the computer.
So if we look at the innovation sequence and we can see that the main is zero centered, then there's
a good chance the field is working correctly.
In some cases, you might see the innovation rise up and become biased.
In these cases, the field is not working correctly or it might diverge.
So we always want to make sure that the innovation sequences for all the different components are zero
main.
The next property that we want to look at is the innovation variance, so we want to make sure that
the variance of the innovation sequence is the same order of magnitude as you predicted, covariance
of the innovation.
So the predictive covariance is the from the matrix.
So the matrix for this case has diagonal elements, our sigma squared X and Sigma squared Y, and these
are actually plotted on the graph here at a time history.
So the green lines here are the plus or minus one sigma of Sigma X and here plus or minus one sigma
of stigma Y values.
So this is how we get these green lines and we want to make sure that the innovation sequence shown
here is consistent with our predicted innovation from the filter so we can calculate the statistical
properties of the innovation sequence from the simulation using a simpler formula as shown here.
And we want to make sure that this sigma value here for the whole simulation is basically consistent
with this sigma.
So from the simulation, we've actually calculated the exposition, measuring innovation and wide precision
measurement, innovation, standard deviations to be 11 and 10 for the X and Y.
So that's these values.
Here are the variances of these blue lines here and here.
So we can see for the X value we have a variant of 11 for the innovation sequence.
We can also see that the one sigma values this green line here is basically at eleven as well.
So it's halfway between zero and 20.
The same thing can be said for the Y innovation.
We have got the innovation level of about 10 and we have a standard deviation of that 10.
So these are basically showing that the innovation sequence is consistent with the actual innovation
predicted from the common filter.
So this is showing that the filter is tuned correctly and it's operating very well.
So now that we've had a look at how to tell if a filter is correctly tuned and working, we want to
have a look at how to actually go about tuning the common filter.
We have already covered one of the important parameters of tuning the filter, which are the initial
conditions.
So we want the initial condition X not to be the best guess or assumption about the initial state of
the system.
Now, this comes through some engineering judgments about what assumptions we can make or what value
the initial step we have when we first start the filter.
And again, we can use the delay, the initialization, to correctly initialize the filter from a known
condition.
We also want to make sure that p not sorry, the covariance, the initial covariance is large enough
to encompass the truth.
So we want to make sure that whatever value we have for Paisner is the variance is large enough that
the truth is contained within that amount around our initial condition.
Not.
Now, the main parameters we're going to use to tune the come and filter are the covariance matrices
for the process model noise and sensor model noise.
So looking at the sensor model noise or covariance, this are matrix here can be generated from information
about the properties of the measurements or the sensor.
So we can look at a data sheet of the sensor.
We could run some tests or experiments on a sensor to come up with some statistical properties of what
the noise characteristics are like in the sensor and in the measurements.
And then we encourage that information into the matrix.
So, for example, let's say we had a sensor and had two measurements, so it said one and said two.
And the measurements are plus or minus a around the truth or plus minus B, the truth for the second
component.
We're using this information here.
We can approximate this inside a Gaussian distribution.
So we could say that our sigma said one or sigma two is just give me three or three B to encourage this
information inside an approximate Gaussian distribution.
Once we have these standard deviations of the Galaxy distribution, we can just throw them into the
diagonal elements of the Matrix.
So we might get this information from a data sheet.
We might make a few approximations and generate our covariance matrix for the sense of noise properties.
The sensory data itself might actually give you the variances of the measurements itself.
It all depends on what information is available and if the information is not available.
You can never run experiments or tests to work out the distribution or we have to make some approximate
assumptions.
And then finally, the last thing we can use to change the filter is the process model covariance and
in general, we want to make the small enough to allow the filter to work in all situations.
If the key metrics is set to small, then we might run into the condition that the it becomes inconsistent.
The variance on the estimate is too small to actually encompass the truth.
So the field becomes inconsistent.
If the relative size of the key matrix to the Matrix in terms of magnitude is too large, it means that
we're injecting a lot of uncertainty into the system.
It means that our uncertainty grows very quickly and it means that our state estimates or the uncertainty
around the state estimates is going to be very large, which is different to what we actually want to
use the common filter for, because we want to use a common filter to obtain better estimates of the
state.
Now we can judge the performance of a common filter.
And if we assume that is correctly tuned and it's consistent, then we can based this performance on
the overall estimation error for a series of runs or the typical performance or the typical estimation
error performance.
So we can say that when we run, this filter is always going to be within plus or minus a certain amount
of the true state.
So this is one way of how to judge the performance.
The next way of judging the performance of the Common Filter is to look at the size of the state variances.
So, for example, if we have a position sensor and the measurements of variance of 10 meters for one
sigma, but the common filter has the position estimates from these measurements to be 20 meters, well,
we can easily see that this filter is pretty useless.
It's just better to use the raw measurements because I have a better accuracy than the actual filtered
estimates.
Now, if we contrast this to the situation where we have a precision sensor giving us our measurements
that have a 10 meter standard deviation, however, the Common Filter estimates have a three made a
standard deviation.
You can see that the opposition estimates from the filter are a lot better than using the raw measurements
themselves.
So this is a good situation to be in.
It means that the current field is working well and it produces better position estimates from the field
than using the raw measurements.
So this is a situation that we typically want to be in.
This is one of the reasons why we want to use the common filter in the first place.
So hopefully now, you know, a bit more information about how to tell if a common filter is chained
and working correctly, how we actually go about tuning the common filter and how we can actually measure
the performance of the common filter.
In the next video, we'll go through the 2D tracker example and actually look at how we can apply this.