In this lecture, we will be looking at the theory behind vector auto, regressive and moving average
models.
So let's begin with the model itself, since I think it's pretty straightforward.
Basically, in the previous Arima model, the Time series was a scalar.
Now the Time series is a vector.
This is because of the Time series now contains multiple observations at once.
For example, the temperature in multiple cities or the voltage at multiple locations in your brain.
So in the scalar case, all the parameters are scalars as well, since the scalar times a scalar gives
you a scalar.
But in the vector case, the multiplicative parameters are matrices.
This is because of matrix times.
A vector gives you a vector.
The bias term, however, is a vector.
Now normally note that I will not use the arrows above variables to denote vectors.
I'm just doing that here to be extra explicit.
But most of the time it should be clear whether or not this is the case.
Also, notice that I use capital letters from matrices.
So in both cases we use five for the auto regressive part and theta for the moving average part.
But in the AMA case we have lower case and lower case theta.
For Varma, we have uppercase Phi and uppercase theta.
OK, so here are some quick things to notice, firstly, note that these matrices are square matrices.
This is because if your time series values are vectors of sized, then you need a D by The Matrix to
get back a vector of sized after multiplying by the Matrix.
Also note, this model still only expresses linear relationships.
As with scalar omma models.
It's possible to have purely auto regressive, purely moving average or both.
The most common model is the vector auto regression, for reasons we will learn about later.
Notice that we still use the parameters and cue the same way we did before.
So is the number of pass values in the Time series and Q is the number of past errors.
Not that these errors are also vectors of sized, the same as the Time series itself.
OK, so I hope you agree that the Vamo model is a pretty straightforward extension of what we saw before
the rest of this lecture, we'll look at a few extra details and we'll try to gain a better understanding
of what the model does.
So the easiest way to understand what the model is doing is to consider of Arpit in two dimensions.
When we do this, it's easy to write down in scalar form to make it simple.
Let's consider of our one.
It's easy to see that our new model is just like our old model, except with extra terms in particular.
The current output depends on its own past lags as it normally would for an error process.
But it also depends on past lags of the other time series as well.
And note that this relationship is completely linear in terms of all the legs, whether they come from
the same or different components.
Now, suppose that we simply had to separate air one models in this case, we see that the equations
would almost be the same, except that there are no cross terms.
So each time series cannot affect the others.
So that's the premise of this model.
By using vector auto regression, you're assuming that there's some predictive capacity across multiple
scale or time series.
One interesting exercise to consider is how many parameters are needed for a vector out of aggression
compared to just multiple scalar auto regressions.
Let's consider only just for simplicity as an exercise.
You might want to try to derive these four Winmar as well.
So in the scalar case, if we have a D independent time series and corresponding IRP models, then in
total we will have D times plus one parameters.
This is because each individual model will have plus one parameters, including the intercept and we
will have a D of these.
Now let's consider the vector case in the vector case.
We will have P matrices each of size D.
We'll also have a bias term of size D, which is now a vector.
So in total we will have P times D squared plus the parameters.
You should convince yourself that as D grows, the number of parameters in the vector model grows faster.
In fact, we would say it grows quite dramatically with the number of components.
You can imagine why this might be detrimental in the case where your dimensionality is large.
So if you've studied machine learning, then you've seen that having too many parameters can lead to
overfitting, which is not good.
This means that your model performs well on in sample data but underperforms on new data.
So one question to consider is, why are purely auto regressive models more popular than Varma where
we combine it with the vector moving average?
This seems counterintuitive since we've seen that moving average terms can make a rhema more powerful.
So the reason has to do with model identifiability.
Now, this can get pretty mathematical, but the basic idea is a Vamo models are not unique, meaning
you cannot uniquely determine pacu.
From a practical standpoint, this just means you'll get a warning from stat's models whenever you try
to fit Varma.
Now, whether or not that matters to you is up to you.
Something else to consider is why is there no such thing as a very well, just like with armor, we
only want to fit a model if the time series is weeks and stationary.
But if this is the case, then why do we use Varma and not Roraima?
Well, to be more specific.
There seems to be some discussion of it online, but you won't find it in stats models.
One reason this might be the case is there some parts of your time series might need to be different
to a different number of times.
So there won't only be one value for D.
For example, if you're looking at GDP, this might grow over time with a trend.
But if you're looking at unemployment at the same time, this would likely not have a trend.
Since this is the case, it will be your responsibility to make sure each component of your time series
is stationary before using Varma.
OK, so now that you understand the basics of Varma, let's think about how to use it in Python, you'll
find that it's pretty similar to a rhema.
We'll look at both VMAX and VARE, which have slightly different APIs for some reason.
OK, so let's start with Varma.
In this case, we'll import a model called VMAX, which allows you to add exogenous data.
This takes in a multivariate time series data set of shape t TBD ends in order tuple.
So there's tuple should contain your values for PACU.
But note that it does not specify any differences since you'll have to do that yourself.
The next step is to call model that fit, which returns a results object.
This takes in some optional parameters, like the maximum number of iterations.
So this isn't super important unless the defaults aren't working for you.
So I encourage you to check the documentation if that's the case.
OK, so as usual, once we have the results object, we can call the fitted values attribute, which
will return the sample predictions.
We can also call get forecast passing in the number of steps to forecast.
As with the AMA, we can also get confidence intervals by using the function content as before.
This returns a data frame with each column name prepend ID with either upper or lower.
The next step is to look at the API for VoIP, which is a bit different.
So in this case, we start the same way as before by creating of our object and passing in a TBD multivariate
time series.
At this point, you do not pass in the model order.
So one interesting feature of this model is that it actually has a function called select order.
When you call this function, you pass in the maximum number of legs you want your model to have.
It then goes through every possibility and computes the AIC along with other criteria, which have a
similar purpose.
As mentioned before, statisticians love having multiple options, but normally we just use the AIC.
So from here you'll get an object of type lag order results from this object.
You can call an attribute called selected orders.
This will return a dictionary where the key is the criterion and the value is the selected order.
So you'll find the AIC back and so forth, each with a number that tells you which order leads to the
best value.
However, note that all this work is not really necessary because you have to pass in the same thing
anyway when you call model does fit in this case, when you call model does fit, you pass in the maximum
number of lags and the information criterion you want to use.
So the fit function is going to do all the same work again.
Note that this also means you cannot choose your own p.
The real use of calling select order manually is that you can get to see if there's any discrepancy
if you choose a different criterion.
OK, so as usual, after you call modeled outfit, this is going to give you back of our results objects.
From this, you can get the order selected by the fit function by calling the attribute K. Underscore
are now this might seem just like an interesting value to look at, but in fact, it's required in order
to complete the following steps.
So the next step is to compute the forecast.
But for some reason, this forecast function is unlike the others we've seen in this case.
This function also takes in the prior values in the Time series, as you recall, other models we've
looked at like AAMA, Arima and Varma do not work like this.
So in order to get the forecast, you'll have to pass in the pass values of the Time series.
This is where the lag order comes into play, since that's how many pass values you need to pass in
in stat's models.
They call this the prior value not to be confused with the prior in Bayesian machine learning.
Now, you might think that this is pretty inconvenient, but in fact it's pretty useful.
Recall that for other models like Arima, if you wanted to incorporate new data, we would just have
to train the whole model again and get forecast again.
Otherwise the model would have to use predicted values.
So, for example, if you want to forecast YFC plus three, you would first need to forecast YFC plus
one and why two plus two and use those predictions to compute the prediction for YFC plus three.
But what if today is T plus two and you just want to make a forecast for one day ahead?
In other words, you want to use the true values for why 50 plus one and YFC plus two?
Well, with the rhema, we would need to train a new model that uses all the data up to date plus two.
But with var we can just pass in the true lag's values assuming that the model hasn't changed.
So this is useful.
Assuming that you don't want to train a new model every day.
You'll recall that this is also the same approach used in regular machine learning.
One final thing to notice here is that both the prior input and the forecast are no higher res.
This is different from the previous examples which accept data frames as input and give back forecast
objects in return.
OK, so let's quickly summarize this lecture since it's been quite long.
We first looked at the equation for the full Varma Piku model.
This helps us recognize that it's just a straightforward extension of Arma two vectors.
We noted that this is still a linear model, except that now each time series depends linearly on its
own, like values and the values of the other time series.
We learn that one problem with Full Varma is that generally these models are not identifiable.
You'll also see that they take longer to train.
We learn that the number of parameters and vector models grows quadratic with the number of dimensions
in the Time series.
This could lead to overfitting and it's due to the fact that our parameters are matrices which are square.
We then looked at the stats models API for both of Varma and VAR, which are slightly different.
We saw that VAR models have the option for automatic order selection, although you could do the same
thing manually with VMAX if you wanted.