In this next part of the course, we are going to begin discussing a different kind of time series model,
which is more like the machine learning type of model that you might be familiar with.
In particular, we'll be looking at auto regressive models, moving average models and combinations
thereof.
Generally, these are known as Arima models.
R stands for auto regressive, Emmas stands for Moving Average and I stands for Integrated.
You will learn what all of these terms mean in the coming lectures.
Note that in this context, moving average does not mean what it meant in the previous lectures, so
don't get them confused.
The names are similar, but as you'll see, the actual models are completely different.
This lecture will focus on auto regressive models specifically.
But first, what is the difference between a Rhema models and the exponential smoothing models we previously
discussed?
As you recall, the exponential smoothing models are built for very specific kinds of data.
They model certain aspects of Time series very explicitly.
In particular, they model linear trends and they model seasonality.
These are built into these models.
On the other hand, you'll find that Arima models impose no such structure in that way.
They are more in the spirit of modern machine learning, where you take a model and you try to fit it
to your data, whatever structure your data may have.
So what are auto regressive models, auto regressive models are basically linear regression models where
the inputs also known as the predictors, are past data points in the Time series.
Let's first review how linear regression works since we may not have gone into enough detail earlier
in the course.
As you recall, the simplest linear regression model looks like this.
We say Y hat equals M, X plus B, in this case, X is the input and Y hat is the prediction and B are
parameters that are found through minimizing the error of the predictions.
One simple example of linear regression would be trying to predict salary from years of experience.
So X would represent years of experience and Y would represent salary.
But what if we have more than one input, you might think years of experience might not be good enough
on its own to make a good prediction about salary.
Let's say we want to take into account age as well.
Now, our model will look like this y hat equals one time zwaan plus W two times X two plus B in this
case, X one could represent years of experience and X to represent age as before.
One, two and B are found by minimizing the error of the predictions compared to their true values in
the data.
Let's think about what the interpretation of one and two and B could be as before B represents the Y
intercept, that is the value of Y when all the X's are zero.
If you plug in X one equals zero and X two equals zero, you can see that this is true.
The WS, on the other hand, tell us how each of the X's affects Y if X one increases by one and everything
else that is X two remains constant, then Y hat will increase by one.
If you don't see this right away, I would recommend plugging in some numbers until you are convinced.
So one is the amount that we had increases by when X one increases by one, to give you a concrete example,
if X one represents years of experience and one equals 5000, then that means in our model, every additional
year of experience should lead to a five thousand dollar increase in salary.
On average.
Note that we sometimes call this model multiple linear regression due to the fact that there are multiple
inputs, whereas before we would call that simple linear regression.
Usually I don't make this distinction and I just call it linear regression.
OK, so what does this have to do with Time series and auto regressive models in particular?
Well, an auto regressive model is nothing but a multiple linear regression model where the inputs are
the past values.
In the Time series, for example, we had a time t is a linear function of Y a time T minus one Y a
time T minus two and so on up to Y a time T minus P.
Note that when we use past data points in the Time series, that is why it time T minus one back to
Y time T minus P.
We call this an AARP model.
Moreover, mostly we say that this is an auto regressive model of order P.
Note that an equivalent way of expressing a linear regression model is this, instead of having Y hat
on the left, we just have Y at time T on the right side, we have added a noise term for time t epsilon
sub t.
What this says is that what do we actually measure that is the true Y is a linear model of the inputs
plus some noise.
This must be the case since as you've seen, usually when we do a linear regression, the data points
do not fall exactly on the line.
Another way of thinking about this is that y had AT&T is just the expected value of Y of T assuming
that the noise terms expected value is zero.
Now, just as a theoretical exercise, we are going to discuss how you might use psychic learn to implement
an auto regressive model.
In actuality, we are going to use stat's models, but I find it very instructive to think about how
you use it, learn which forces you to think about the structure of the data.
Let's think about what a data set might look like for a normal linear regression model where I'm trying
to predict salary from years of experience and age.
In order to do this, I might do a survey where I ask everyone in the office to anonymously fill in
a spreadsheet that I've created.
After all of my coworkers have filled in the spreadsheet, I will have something that looks like what
you see here.
Each row corresponds to a different person.
Each column represents an input feature in our case.
That's years of experience and age.
We call this big table X.
In fact, it is indeed a table, but it is also a matrix.
Furthermore, we have Y, which is just a single column of salaries.
Obviously each of these salaries corresponds to the same row in the X matrix.
So if the first row of X belongs to Alice, then the first row of Y is Alice's salary.
Using this knowledge, can we build a linear regression data set for an AP model?
The answer is yes.
Well, I'm just going to show you the answer, and your job will be to check that this makes sense.
In fact, you could even implement this yourself using Scicluna as an exercise.
So the idea is this.
Suppose that we start with the fact that there are 10 data points.
Why one up to why ten?
We know that the equation to predict Y four is a linear function of Y one way too.
And Y three.
Therefore, if Y four is the target then y one y two and Y three are the inputs.
Similarly, if Y five is the target then Y to y three and Y four are the inputs.
If we keep doing this, we get the table that you see here.
Note that in machine learning, we typically say that the size of X is envied where and is the number
of samples and it is the number of input features for arena models.
We will stick with the convention that the number of predictors is P.
So in case you are confused about this, just remember that in the context of a Remar D is equal to
P.
So at this point, you know how to convert your time series into a tabular data format that can be passed
in the cycle, learn once you have this, the rest is easy.
As usual, you can create a model of type of linear regression you call model that fit to fit your model.
After that, you can call model that predict to make predictions or model that score to check how good
your model is.
This is just another instance of my rule.
All data is the same.
It doesn't matter what your data is.
The basic API never changes whether you're predicting salary or you're predicting a time series.
Another interesting topic to consider is this we know based on our knowledge of machine learning, that
linear models are not that powerful.
In fact, they can only represent lines or planes.
In addition to my rule, all data is the same.
I have another rule.
All machine learning interfaces are the same.
What does this mean?
It means that most of the time when you were doing machine learning, you can use different models say
was learn without any change to the rest of your code.
Previously we used linear regression and we noted that this is an auto regressive model.
But why should we be limited to just linear regression?
What's stopping us from, say, using a neural network or random forest, which we know are much more
powerful models?
If we have our X and we have a Y, there was nearly no change to the code required.
Simply replace the linear regression class with some other class.
On the other hand, it's also important to remember that in this class, which is focused around financial
data, there was a strong emphasis on the modeling aspect rather than predictive accuracy.
What you will learn in the following lectures is that Arima models allow us to understand our data in
a much more in-depth way than simply plugging in a neural network.
Yet another way to understand auto regressive models and Arima in general is this, as you know, one
of the major themes of this course is price simulation.
What were you actually studying in?
This course is random processes and naturally a random process can be simulated using a computer.
We looked at a few examples in previous lectures, such as simulating random walks and simulating a
sequence of coin flips to decide whether you would take a step to the left or take a step to the right.
So it's useful to ask the question if we are given an AP model that is all of the weights of the model,
how do we generate a time series?
Not that this is kind of the opposite question that we are used to considering in machine learning and
machine learning.
The question we usually ask is, given the time series, tell me the weights.
Now we are going in reverse order.
Given the weights, tell me what the time series might look like.
So how can we do this, let's suppose we have an R three model, so the next value in our Time series
always depends on the last three values.
Let's suppose also that the first three values are given so that the first value we are responsible
for generating is Y for in this case, the first step will be to generate Epsilon for the error at time
four, which is a normal with mean zero and some variance sigma squared.
Then we can use our formula to calculate Y for next.
We have to calculate Y five.
Before we can do that, we first generate Epsilon five again from the same normal with mean zero and
variance sigma squared.
Then we add Epsilon five to the Y to Y three and newly generated Y four terms and that gives us Y five.
Then we generate Epsilon six and Y six, epsilon seven and Y seven and so on and so forth.
So you see that by doing this we understand that this model is more expressive than a simple random
walk.
This would be a random walk if we only had a one previous value and a weight of one.
Instead, we have a bunch more terms and arbitrary weights.
In the next few lectures, we will make this model even more expressive.