So in this lecture, we'll be discussing the relationship between state space models and simple ordnance.
Let's begin by reviewing the state space model so that we begin this lecture at a common point.
OK, so the state space model is a linear model that specifies how some hidden state vector X of T is
computed from the previous head and state vector X of T minus one.
And some control input vector U of T.
Note that these are related by matrices by convention.
We call these matrices A and B.
We also have another equation telling us how the hidden state is related to our observation vector y
of T sometimes y of T can also be directly affected by the control input U of T.
But very often this is simply left out in the second equation.
By convention, we call the matrices C and D.
So just as a quick reminder of how such a states based model might be used in the real world.
Recall that this is the set of equations that we use in control systems.
An example of a real world problem is that the inverted pendulum which you've seen if you've taken my
courses on reinforcement learning, what's cool about typical courses on control systems is that unlike
reinforcement learning, we often get to work with real world projects.
So in the image here, you can see an example of a live physical carpool system that can be controlled
using the techniques of control theory.
So as an example of how this relates to our equations, we might represent our state vector with four
measurements which are horizontal displacement, horizontal velocity angle from the vertical and angular
velocity.
What we get to observe might only be the displacement and angle.
Since measuring velocity could be more difficult in practice because this system is based on the laws
of physics.
We can use physics equations to determine the matrices A, B, C and D.
OK, so now let's recall the equations for a simple reason, suppose that we also include the output
layer as well, and we assume that this Arnon is many too many, meaning that we compute the output
for every time step.
Note that for simplicity, I've excluded bias terms.
In this case, our hidden state vector is called HFC, which is dependent on the previous head and state
vector of T minus one.
And the model input X50.
They are related by the weights hidden and W input with an activation function sigma, which can be
the real you 10h and so forth.
Furthermore, the output y of T is related to the head and state h of T using the weight matrix W output.
Of course, at this point, it should be obvious that the states base model in the Arnon are pretty
much exactly the same, except that we've renamed some variables in the aunt and has a nominee already.
As such, we can conclude that the ANA is just a non-linear generalization of the states base model.
In addition, using the techniques of machine learning, this gives us another way to learn the matrices
A, B, C and D without having to derive physics equations ourselves.