In this lecture, we are going to look at the code for the exponentially weighted moving average.
As usual, you can look at the title of the notebook to determine what notebook we are currently looking
at.
So before we start to look at the code itself, let's talk about the data set in this lecture and for
most of the lectures in this section, we'll be using the airline passengers data said.
This is a famous time series analysis data set, and it's basically the mists of Time series analysis.
That said, the first line here is to download our airline passengers CSFI.
Next, we import pendas and non-pay, not too surprising.
Next, we use prediabetes, D.V. to load in our CSFI.
Again, not too surprising.
Notice that the indexed column is month.
So this is monthly data rather than daily data.
Next, we do a different head to see the first few rows of our data set.
As you can see, our index is the month and the passengers column contains the number of passengers
in that month.
Next, we do advocate is in L.A. to check if there are any missing values.
Since the summer zero, there are no missing values.
Next, we do a plot to plot our data said.
So let's spend some time analyzing what we are seeing, first of all, this looks nothing like a typical
stock price time series.
There seems to be very little randomness in this data set.
In fact, that would mean that this is a great candidate for statistical in machine learning techniques.
Remember that we used to call machine learning pattern recognition.
That is, machine learning is all about recognizing and modeling patterns.
And clearly there is a pattern here.
On the other hand, stock prices might be characterized by their lack of patterns.
We can see that with this data set, there seems to be a trend component and the cyclical component
of the trend means that the signal is generally going upwards.
The cyclical component means that there's some periodicity where the same pattern repeats itself.
We can see that this cyclical pattern actually amplifies over time, so it's not a constant cyclical
pattern.
It's also difficult to tell whether the trend is increasing linearly or if that is also increasing over
time, perhaps exponentially in this Time series.
We don't have enough data to see that.
However, one might surmise that many real world phenomena grow exponentially.
For example, the number of transistors on an integrated circuit grows exponentially.
This is known as Moore's Law due to recent world events.
We know that viral transmission exhibits exponential growth.
Of course, this is limited at some point since the world population is finite.
So in fact, the overall curve might look like a sigmoid.
Let's begin by setting our alpha parameter to zero point two.
Why did I choose a zero point to no particular reason you can think of this as a hyper parameter that
you can optimize?
Next, let's calculate the WEMA, this can be accomplished by first using the IWM function we pass in
our alpha parameter and we say adjust equals false y adjust equals false.
When you set adjust equals true, it actually does a different calculation than what we discussed in
the three lectures, which is not what one usually considers to be the WEMA.
So we use adjust equals false so that Pendas uses the method we expect.
Finally, we call the mean function because we want the exponentially weighted moving average, which
corresponds to the exponentially weighted mean.
We assign the result to the column called a.
Next, just in case you're interested in the data type returned by the EWTN function, I've printed
that out.
So as you can see, we get back an object of type IWM, so if you're interested in looking at the documentation
for this object now, you know which page to go to.
Next, we do a plot to plot everything currently in our data frame, as you can see, the exponentially
weighted moving average is kind of like a delayed and adaptive version of the original data set similar
to the simple moving average.
You might want to play around with the value of Alpha to get a better feel for how it affects the moving
average.
Next, what I always like to do if the equation is simple enough, is to check whether Pendas is doing
what we expect it to do.
That is, does it implement the formula we discussed in the theory lecture to check this?
We're going to use that formula manually to start.
We're going to create an empty list called Manual WEMA.
Next, we're going to look through every element in the passenger's column of our data set inside the
loop.
We're going to check whether our manual GWM list is empty or not.
Remember that the IWM is calculated based on its previous value.
If there is no previous value, then we can't use the formula.
Of course, that occurs on the very first value.
The question is how does it WMA initialize itself?
In fact, there are many answers to this question you may have seen in my machine learning course is
that the typical answer is to set the first value to zero.
This induces a bias in the moving average because if, let's say all your actual values are above one
hundred, as they are in this case, the average is going to get pushed down from one hundred to zero
as if zero were actually part of the data set.
There is a way to correct this using a method called bias correction, which is used in modern optimization
algorithms such as Atem.
However, that's outside the scope of this course for us.
We will simply use the method of copying the first value.
Otherwise, when the length of manual WMA is greater than zero, we simply use our formula.
We assign the result to X hat and then we append X had to our list.
Finally, when we are outside this loop, we assign our list to a new column of the data frame called
Emmanuel.
Next, we do a doughnut plot again.
We see that, indeed, our manual calculation appears to match what Panis does.
Now you have a much better understanding of what Panis is doing under the hood.
Not that we can also do a detached head to check how close the values actually are from the small print
out, they appear to be exactly the same.
Now, since these two columns are exactly the same, the next thing I'm going to do is remove the manual
column since it's redundant.
This will help us clean up our data frame for the next few lectures.