In this lecture, we are going to be looking at stationary in code specifically will be applying the
augmented Dicky Fuller test to a variety of data sets and will learn whether or not it actually fits
our intuition about what is stationary and what is not.
This lecture is going to walk you through a prepared CoLab notebook, although a very good exercise,
which I always recommend, is once you know how this is done, to try and recreate it yourself with
as few references as possible.
As always, you can check the lectures, how to code by yourself and how to practice for a more in-depth
discussion.
If there's anything in this lecture you didn't understand or you think I missed a step or didn't explain
why we were doing something, please use the Q&A to inquire.
As usual, you can look at the title of the notebook to determine what notebook we are currently looking
at.
OK, so let's start by importing pendas, numpty, matplotlib and the functionality fuller from stat's
models.
Next, we're going to download our airline passengers, Data said.
Next, we're going to load in our data set using pedigreed CSFI.
Next, we're going to plot our data said.
So this will help remind you that this data set is not stationary, there's both a trend and a seasonal
component to this data.
Next, we're going to test out the 80 full or function UNDEF passengers.
OK, so as you can see, we get a big table of numbers.
What do these numbers mean?
Well, we can check the documentation for stats models.
The two key pieces of information we are interested in are the test statistic and the P-value.
Most of the p value, as you know, the test statistic is used to compute the P value.
Note that these are the first to return values in the tuple.
So next, we're going to write a little helper function called ATF, this will help us more easily check
the results of the ATF test rather than trying to interpret a couple of numbers.
This function takes in a series called X.
Inside the function, we run the fuller function on X. Next, we print the test statistic and the P
value.
As is typical, we'll use a significance threshold of five percent if the P value is significant.
We'll print out stationery.
Otherwise we'll print out non stationary.
OK, so let's test their function on the DF passengers column once again.
As you can see, the p value is quite large and we do not reject the null hypothesis.
In other words, we conclude that this Time series is non stationary.
Hopefully this is the result you anticipated.
Next, let's test whether or not this function works for an actual stationary signal.
So this will be ID noise sampled from the standard normal note that the signal is strong and stationary
because the entire distribution remains constant over time.
All right, and we get a very small p value on the order of ten to the minus 18, therefore we conclude
that the signal is stationary as expected.
Now, everyone knows about the normal distribution, but what if we try a more exotic distribution like
the Gamma?
Again, A theoretically we know that the signal is strong, sent stationary because the distribution
does not change over time and each sample is Eid.
OK, and we get another very small p value on the order of ten to the minus 16.
Again, we conclude that this signal is stationary.
And by the way, if you don't know what the gamma distribution is, I'd recommend plotting this Time
series and also plotting a histogram of these samples.
The gamma is a very important distribution, especially in Bayesian machine learning.
Next, we're going to take the log of the passengers call, we'll be making use of this over the next
few experiments.
So next, let's run a test on the log passengers column.
Not surprisingly, the result is that it's still non stationary.
Next, let's difference the passengers call them.
Remember that this is what we would like to fit in a rhema, too, so if we plot the this column, we
see that the variance of the signal appears to increase over time.
We can therefore guess that the signal is nine stationary.
All right, so let's run the ATF test on the first difference of the passengers, call them, note that
we have to drop in because whenever we take the first difference, we always have one and a value in
the first row.
OK, so, again, we end up saying that the signal is non stationary.
But notice how close the P value is to five percent.
If you had set your significant threshold to a higher number, like 10 percent, you would have concluded
that the above signal is stationary.
OK, so next, we're going to take the first difference of the log passengers will store this in a column
called Dif Log.
If we plot the flag, we notice that unlike the first difference of the raw passengers, the variance
here does not increase over time.
Perhaps then this signal is more stationary than just the Rothfuss difference, of course.
We still have to run our test.
So next, we run the ATF test on the deflate column again, recall that we have to drop the NRA values.
All right, so surprisingly, we get a higher p value than we did when the variance was increasing.
OK, so this is a good opportunity to remember that we care about stock prices in this cause, so let's
download the SP 500 a CSV.
Next, we're going to load in the S&P 500 at CCV using Peaty that reads, CSFI will assign this data
frame to a variable called stocks.
Next, we do a stock stat head just to remind you of what's inside this data frame.
As you can see, the final column is the name column, which can be used to filter out any particular
stock ticker.
Next, we're going to grab all the rows where the name is equal to Googs and we're going to grab the
close column.
Next, we're going to take the log of the close, call him to get the log price.
Next, we're going to take the first difference of the log price column to get the log return.
Next, we're going to plot the log price as a time series.
As you can see, there is a clear trend.
Next, we're going to plot the log return.
So this looks pretty stationary, although the variance does seem to increase in some places.
OK, so let's try the ATF test on the log price.
As expected, we conclude that the log price is not stationary.
Next, let's run the ATF test on the log returns.
So the log returns are so stationary that the P-value just gets rounded down to zero.
Now, just for good measure, let's test a different stock, let's use our other favorite stock, Starbucks,
again, the same code to calculate the log price and the log return.
Next, we plot the log price.
Again, we see a pretty clear trend.
Next, we plot the log return again, it looks pretty stationary, except that the variance seems to
not be constant.
So let's run the ATF test on the Starbucks log price.
As expected, it's nine stationary.
Next, let's run the ATF test on the Starbucks log return.
All right, so just like the Google log returns, the p value is so small that it gets rounded down
to zero.
Therefore, we conclude that these log returns are stationary.