In this lecture, we are going to combine what we learned about in the previous lectures and build up
the full Arima model.
Before we do that, however, let's first discuss what we get when we combine the R and the M models
only, which gives us the AMA model.
After that, we'll discuss what the AI component means and then combine that with the AMA model to give
us the Arima model.
So I think the AMA model is pretty straightforward once you understand the auto of aggressive and moving
average parts separately, the AMA model simply adds them together.
Therefore, you would use this model if you believe that each point in the time series is linearly correlated
with both pass points in the Time series, as well as past errors of the model.
Note that the abbreviated form of this model is ama p.
Q This is short for auto regressive moving average model with the auto regressive part has auto P and
the moving average part has autocue.
Now that we know what Amah is, let's talk about Arima, the IPART part in Arima stands for Integrated.
Therefore, the full name for this model is auto regressive, integrated, moving average.
What is interesting about this model is that the integrated part has no parameters.
So the way it looks is very different from the auto regressive and moving average parts.
So how does the integrated part where.
To understand the integrated model, we have to understand differences, I suppose that we have some
time series Y of T to perform differences on this Time series, you would define a new time series,
Delta Y of T, which is defined as Y of T, minus Y of T minus one.
That is for each point in the Time series we subtract of the previous data point.
Now a good question to ask is, why would you want to do such a thing?
In fact, this is not the first time we have seen differences in this course.
We have seen it twice previously.
One time we saw different.
Same is with log prices.
When we perform different single log prices, we get the log return.
As you know, this is a pretty important quantity in finance.
Another time we saw different is what the whole winters' model, the first difference of the level is
used to estimate the trend of the series.
In other words, one way to think of differences is defending, or in other words, separating out the
trend.
In fact, we see this in Stock Price Time series as well.
After taking the first difference of the log prices, we get the log return, which is generally a noisy
signal that fluctuates around a value close to zero.
So why do we want to difference our Time series, the reason for this is when we use Arma models, that
is auto regressive, moving average models.
We want the time series.
We use the model on to be close to stationary.
Now, what does stationary mean?
Again, at a high level?
It means that the distribution of your signal does not change over time.
There are more details we can discuss regarding stationary, but we will leave this for a later lecture.
Stationary is a nice property because it means that various statistics, such as the mean variance and
auto correlation, will remain constant over time.
This is a good thing when you're trying to fit an arm, a model, because remember that each window
of the Time series is like another training point when you are fitting the model.
Now, this slide is optional, but I want to elaborate on the previous point a bit more.
Recall that earlier I talked about training your own auto regressive model, using Saikat.
Learn by first building your data set from the Time series.
As you can see, this data set is built out of sliding windows from the original time series.
First, the window covers Y one y two and why three.
Then we slide it over one step.
Now the window covers y two y three and Y four, then we slide it over one step again and so on.
One important aspect of machine learning is that the data in your data set is assumed to all come from
the same distribution.
If it didn't, then you wouldn't be able to learn anything.
Remember, machine learning is pattern recognition and pattern recognition is machine learning.
If we mess up the pattern, then we mess up the machine learning model.
Imagine, for example, that we had our table of salary data.
Suppose that we work at a software engineering company and all of the coworkers who helped you fill
in your spreadsheet were software engineers.
Then all of a sudden, one of your buddies from the marketing department fills in their data on the
spreadsheet.
But unfortunately, marketers get paid at differently than software engineers.
And so it's likely that the distribution for marketers salaries is different from the distribution for
a software engineer salaries.
Your model might see this data point as an outlier, which might bias the results.
So that's why we want all of our data to come from the same distribution.
Otherwise, what we are trying to learn won't make much sense if we're trying to predict software engineer
salaries.
We don't want data from people who are not software engineers.
For us, we have a time series and each position on this Time series makes up a new data point in our
training set.
If we would like all these data points to come from the same distribution, then the Time series being
stationary accomplishes that because that's exactly what stationary means.
So let's get back to stationary now, we know why stationary is good.
Well, it turns out that different seeing a time series often helps to make the Time series stationary.
Now, remember that when you're working with real data, nothing is exact and you're always approximating.
Yes, we know about volatility clustering.
So we know that the variance of the log return can fluctuate over time.
Does that mean we throw out a rhema completely?
The answer is no.
And sometimes we may even find that we have to difference twice in order to get a time series that looks
somewhat stationary.
Usually, however, we don't difference more than twice.
OK, so what does defensing have to do with a Arima?
We say that a process is ID if it is stationary after being differenced D times and ID process is a
process that is integrated to order D.
Once we have this, we can define the Arima model.
The Arima model has three arguments P and Q, as you recall, P refers to the auto regressive part.
Q refers to the moving average part and now we know that D refers to the integrated part.
Thus Arima P.
Q is just a model where we have difference.
The original time series D times before applying the ama p q model.
So that's why the eye part of the Arima is strange.
Unlike the auto regressive and moving average parts which have some formula that you can use to make
predictions, the integrated part describes an operation that you perform on the data.
Now that we know what the full Arima model is, we can see that all of the previous models we've discussed
are actually just special cases of the Arima.
For example, a remap zero zero isn't IRP process.
This is also the same as an arm up zero.
Similarly, Arima zero zero Q is an AMA zero.
Q Which is also an Macu Arima zero zero is an ID.
One interesting special case is Arima zero one zero or equivalently, I one, this is what is known
as a random walk, which we discussed previously.
This model looks as follows.
It says that Delta Wafty is equal to Epsilon of t the noise at time t.
Why is this a special case?
Well, it just means that there is no auto regressive part and no moving average part by omitting these
two.
All that we are left with is noise, the left side delta.
It is just the difference.
That Time series, equivalently it says that Y of T minus Y of T minus one is equal to the noise at
time.
T if we move the Y of T minus one it to the other side we get that wave T is equal to Y of T minus one
plus Ypsilanti, which is exactly the random walk formula we discussed earlier.
It says that whatever process in nature leads to our Time series, it takes the existing value Y of
T minus one, generates some random noise Ypsilanti and adds them together to get the next data point
y.
T.
Another way to view this, if we think about financial quantities is this suppose our data set is a
time series of log prices, then the first difference is the log return.
A random walk says the log return is just Ypsilanti, the noise.
What this tells us is that there was nothing about the log return that is predictable except for the
expected value of the noise.
It does not depend on any past values, which is what an auto regressive model would tell us.
It does not depend on any past noise terms, which is what a moving average model would tell us.
Instead, a random walk says that the log return is purely noise from a single point in time and hence
completely unpredictable.
Aside from the mean of the noise, therefore, if we later fit in a rhema model to a stock price time
series and we find that the best RMI model is Arima zero one zero, this tells us that the data follows
a random walk and the return can't be predicted using previous values in the Times series.