OK, so in this lecture, we are going to look at how to implement, walk forward validation and code.
Let's start by downloading our data set, which will be airline passengers.
The next step is to update stats models, so we have the latest API.
The next step is to import the modules we need for this script.
So basically the only new thing is just better tools.
This is useful for looping through all possible combinations of the options we want to try.
The next step is to load in our CEV using pedigreed CSFI.
The next step is to set the frequency of our data from index to mutt's.
The next step is to check the size of our data frame, so this is useful to make sure we don't go too
far back when we create our train set, since we want our train set to have enough data.
The next step is to set a few parameters for our test was that the forecast horizon to be 12 and the
number of walk forward steps to be 10, the effective validation period, which I've called and test,
is the length of the whole data frame, minus H, minus the number of steps, plus one.
You might want to draw this out on paper to make sure it makes sense.
There will be some debugging code in our walk forward function to check this.
The next step is to set some configuration options to try, so please read these and review the documentation
if you need to.
The only thing that's mildly confusing here is the use box Cox list of options.
So according to the documentation, you should be allowed to use the string log as an option.
However, this actually fails.
As you can see from the stats Morse code, it returns an error if you do not pass in true false or offload
value for lambda.
So in fact, passing in the string log is incorrect.
And just to be sure, this is the latest version of the stats model's documentation.
You might want to double check it yourself to confirm what I'm saying.
In any case, the simple solution is to just pass in zero since Lambda Equal Zero corresponds to the
log transform.
OK, so the next step is to implement our walk forward function, it's going to accept as input and
argument for each of the options and the final flag for whether we want to debug.
Inside the function, we'll start by creating a list to store the errors from each round.
We'll also have some useful variables for debugging.
Seen last is a flag that will tell us whether or not we've looped up to the final row in the data frame.
This will ensure the limits of our loop are correct.
We also have a counter called Steps Completed, so we'll increment this by one on each round and by
the end of the loop it should equal to the number of steps we said earlier.
The next step is to enter a loop again, you can check the limits of this loop by drawing them out on
paper and we'll also print out some debugging variables to ensure its correct.
Inside the loop, we index our train and test sets.
Note that unlike what you might think of when we describe this function in words, there's no actual
need to add data to the train set on each round.
Instead, we can just index the data frame using the correct indices.
Furthermore, indexing the data frame just creates a view without needing to create a new data frame
so it doesn't take up any or computation.
The next step is to check if the final index of the test set is equal to the final index of the original
data frame.
If this is true, we set scene last to true.
Again, this is for debugging to ensure that we've looped through the end of the data.
The next step is to increment steps completed by one again for debugging, the next step is to instantiate
our model and call the fit function using the options we passed in.
The next step is to grab our forecast and compute the mean squared error with the test, that we then
append this error to our list of errors.
The next step is to check the debug flag, if this is true, then we print out the value of C and last
and steps completed at the end of this function, we return the mean of the errors list.
OK, so the next step is to just test our function, to make sure that it works well, pass in debug
equals true to make sure our loop limits are correct.
OK, so luckily, scene last is equally true and the number of steps is equal to what we said earlier,
this means our calculations were correct.
OK, so the next step is to figure out how to loop through every combination of our many lists of options.
So one option to do this would be to just create multiple nested for loops.
Of course, this is very ugly and inflexible, since you might want to add new options in the future.
If you have too many options, your lips are going to go off the page, which isn't very nice.
So we're going to use Aitor tools, DOT products, which does exactly what we need, it loops through
every possible combination of items from multiple lists.
Now, the usual way to call this function would be to pass on the list directly into the product function.
However, because the number of lists we have is so long, they would go off the page.
And so an equivalent way of doing this is to pass in a tuple of the arguments we want to pass in and
then use the star.
OK, and so inside this loop will print out whatever this thing gives us to make sure it's what we expect.
OK, so you can see that this is exactly what we expect, every item we get back from this function
is a tuple of the options in different combinations.
OK, so the next step is to finally put this all together and use our walk forward function to find
the best set of options.
We'll start by initializing the best score to infinity and the best options to none.
Then we'll enter a loop that will iterate through all the possible combinations of options inside the
loop.
We'll call our walk forward function with the current set of options.
Again, remember that we don't actually have to expand the tuple of options in order to pass these into
the walk forward function.
We can just use the star.
So from this we get back a score and if this is less than our current best score, we save the score
and we save the options.
OK, so you can see that when we run this, we get some warnings about overflows.
This is probably not an issue since it just means we'll get a bad model.
OK, so the last step in this notebook is to check what the best score and best set of options actually
are.
So we see that the best set of options is a multiplicative trend and multiplicative seasonality.
This makes sense since the seasonal pattern seems to grow over time for airline passengers.
We have true for the downtrend, which is interesting since it doesn't seem like the trend is dampening
over time.
We have legacy heuristic for the Init method, which I guess could go either way.
And interestingly, we have false for use box cox, which means that we got the best results by not
doing any transforms.