1 00:00:00,000 --> 00:00:02,529 Hello, everyone. And welcome back. This is Thistleton and Sadigov. 2 00:00:02,529 --> 00:00:07,139 Today, we're going to continue with ARIMA processes. 3 00:00:07,139 --> 00:00:11,073 So, until now we have seen autoregressive processes, 4 00:00:11,073 --> 00:00:15,060 we have seen moving average processes and we also 5 00:00:15,060 --> 00:00:19,250 have seen mixed ARIMA processes which means that in those models, 6 00:00:19,250 --> 00:00:24,575 in those processes, there are some autoreggressive terms and some moving average terms. 7 00:00:24,575 --> 00:00:26,594 And today, we are going to describe autoregressive, 8 00:00:26,594 --> 00:00:29,760 integrated, moving average models. 9 00:00:29,760 --> 00:00:32,869 In other words, we will have one more addition 10 00:00:32,869 --> 00:00:35,024 to the-- one more update to our previous model. 11 00:00:35,024 --> 00:00:37,523 Autoregressive part, moving average part, 12 00:00:37,523 --> 00:00:41,079 and there will be some integrated part in the middle. 13 00:00:41,079 --> 00:00:45,000 We will learn how to rewrite autoregressive, integrated, 14 00:00:45,000 --> 00:00:46,740 moving average models, in other words, 15 00:00:46,740 --> 00:00:50,850 ARIMA processes using backshift and difference operators. 16 00:00:50,850 --> 00:00:52,832 Let's remember, ARMA processes. 17 00:00:52,832 --> 00:00:58,414 ARMA (p,q) process is defined as the following. 18 00:00:58,414 --> 00:01:00,549 There are P terms. 19 00:01:00,549 --> 00:01:04,739 These P terms are autoregressive terms because XT is 20 00:01:04,739 --> 00:01:09,269 regressed on previous P values of the same time series, 21 00:01:09,269 --> 00:01:17,140 and also XT depends on previous Q noises and then there is a noise for the current time. 22 00:01:17,140 --> 00:01:22,829 So this part is the moving average terms and this part is autoregressive terms. 23 00:01:22,829 --> 00:01:28,229 And we learn how to write this as a polynomial, in polynomial location, 24 00:01:28,229 --> 00:01:30,828 so we can put autoregressive terms in 25 00:01:30,828 --> 00:01:34,099 the left and keep the moving average terms on the right, 26 00:01:34,099 --> 00:01:38,257 and we will have this notation phi(B)Xt, beta(B)Zt. 27 00:01:38,257 --> 00:01:41,405 Phi(B) here is autoagressive polynomial, 28 00:01:41,405 --> 00:01:45,193 beta(B) here is moving average polynomial. 29 00:01:45,193 --> 00:01:51,036 Now, if you think of a Z as a complex number. 30 00:01:51,036 --> 00:01:56,025 We would like to have beta(z) and phi(z) has roots, 31 00:01:56,025 --> 00:01:59,540 complex roots that lie outside of the unit circle so 32 00:01:59,540 --> 00:02:03,904 that our process will be stationary and invertible. 33 00:02:03,904 --> 00:02:08,830 Of course, not every real life dataset time series is stationary. 34 00:02:08,830 --> 00:02:14,419 Sometimes, we do have some non-stationary time serieses and it is possible that 35 00:02:14,419 --> 00:02:21,185 this non-stationarity comes from the systematic change in the trend of time series. 36 00:02:21,185 --> 00:02:26,180 So what we would like to do first before we try to fit ARMA process, 37 00:02:26,180 --> 00:02:29,475 ARMA models into our times series, 38 00:02:29,475 --> 00:02:32,382 I would like to somehow remove the trend. 39 00:02:32,382 --> 00:02:35,427 So how are we going to remove the trend? 40 00:02:35,427 --> 00:02:38,166 We are going to use difference operator, 41 00:02:38,166 --> 00:02:41,955 which is basically one minus backshift operator B. 42 00:02:41,955 --> 00:02:45,289 So, let me just go back to the random walk model that we have 43 00:02:45,289 --> 00:02:49,400 seen at the beginning of the course. 44 00:02:49,400 --> 00:02:54,289 So, if I subtract Xt minus Xt minus one, 45 00:02:54,289 --> 00:02:55,759 that's the difference operator. 46 00:02:55,759 --> 00:03:00,645 I'm basically, I can basically write Xt minus one as (B)Xt. 47 00:03:00,645 --> 00:03:03,395 So this can be written as one minus (B)XT. 48 00:03:03,395 --> 00:03:05,925 So for example, if you look at the random walk which is 49 00:03:05,925 --> 00:03:09,650 basically previous step plus some noise then we can take 50 00:03:09,650 --> 00:03:12,770 this Xt minus one to the left and we can write this as delta Xt 51 00:03:12,770 --> 00:03:17,979 equals to Zt which becomes a stationary process. 52 00:03:17,979 --> 00:03:23,419 So, if the process Xt is autoregressive, integrated, 53 00:03:23,419 --> 00:03:31,479 moving average of order (p,d,q) realize that we have now a new parameter D. Then, 54 00:03:31,479 --> 00:03:33,050 what we mean is the following. 55 00:03:33,050 --> 00:03:39,944 We have this Yt which is Xt applied with the difference operator D many times. 56 00:03:39,944 --> 00:03:42,990 In other words, 1 minus B to the dXt. 57 00:03:42,990 --> 00:03:46,525 This Yt, after we difference the time series, 58 00:03:46,525 --> 00:03:51,754 D many times then Yt is an ARMA process with order p and q. 59 00:03:51,754 --> 00:03:56,030 So whenever Yt here is ARMA(p,q) then Xt, 60 00:03:56,030 --> 00:03:59,194 the original time series is ARIMA(p,d,q) 61 00:03:59,194 --> 00:04:04,759 and d is the number of times that we take the difference. 62 00:04:04,759 --> 00:04:11,668 In other words, we can write this Arima process in the polynomial notation. 63 00:04:11,668 --> 00:04:12,844 We're going to have phi(B), 64 00:04:12,844 --> 00:04:15,919 instead of Yt we have delta d, 65 00:04:15,919 --> 00:04:19,935 difference operator d many times equal to beta(B)Zt. 66 00:04:19,935 --> 00:04:21,290 Or we can write this, 67 00:04:21,290 --> 00:04:23,810 instead of the delta as a difference operator, 68 00:04:23,810 --> 00:04:26,125 we can write this as one minus B, 69 00:04:26,125 --> 00:04:27,529 B being the backshift operator. 70 00:04:27,529 --> 00:04:32,954 Now, usually this differencing-- order of differencing is not too much. 71 00:04:32,954 --> 00:04:36,170 You usually take one or two differences. 72 00:04:36,170 --> 00:04:39,470 Taking over differencing may introduce 73 00:04:39,470 --> 00:04:44,735 the artificial dependence which were not existed in the first place. 74 00:04:44,735 --> 00:04:47,884 We're going to look at our ACF. 75 00:04:47,884 --> 00:04:52,709 ACF will itself might also tell us that maybe differencing is needed. 76 00:04:52,709 --> 00:04:56,089 Realize that if you look at the polynomial phi(z) one minus z 77 00:04:56,089 --> 00:04:59,480 equal to D. Even though phi(z) might 78 00:04:59,480 --> 00:05:08,209 not have a complex root inside the unit circle including the boundary, 79 00:05:08,209 --> 00:05:14,464 one minus z to the d has a unit root with multiplicity of D. So in other words, 80 00:05:14,464 --> 00:05:18,259 ACF of this process, 81 00:05:18,259 --> 00:05:21,134 if there has to decay very, very slowly, 82 00:05:21,134 --> 00:05:24,845 so once you see very slow decaying ACF that 83 00:05:24,845 --> 00:05:28,490 is also a suggestion that maybe we have to do some differencing. 84 00:05:28,490 --> 00:05:36,805 Now, later on, we will going to actually modeling real life data sets. 85 00:05:36,805 --> 00:05:40,144 So, basically, we're going to go this checklist in a way. 86 00:05:40,144 --> 00:05:42,134 This is going to be our guide. 87 00:05:42,134 --> 00:05:44,839 If there is a trend that will suggest a differencing, 88 00:05:44,839 --> 00:05:47,665 if there's a variation in the variance. 89 00:05:47,665 --> 00:05:50,750 Right. If the variance is different in one part of 90 00:05:50,750 --> 00:05:55,040 the time series from the other part of that time series which means it's not stationary. 91 00:05:55,040 --> 00:05:58,325 We have to use some kind of transformation to stabilize 92 00:05:58,325 --> 00:06:02,675 the variance and the common transformations are lower at them. 93 00:06:02,675 --> 00:06:05,300 And then, sometimes, we will need the differencing. 94 00:06:05,300 --> 00:06:07,819 So if you take the logarithm and then differencing, 95 00:06:07,819 --> 00:06:09,553 that whole thing is called, 96 00:06:09,553 --> 00:06:15,865 in financial time series they call it a log-return of the time series. 97 00:06:15,865 --> 00:06:17,088 We're going to look at ACF, 98 00:06:17,088 --> 00:06:22,115 autocorrelation function which might suggest [inaudible] for us. 99 00:06:22,115 --> 00:06:28,509 We're going to look at PACF of the difference or transformed time series. 100 00:06:28,509 --> 00:06:34,100 And that might suggest for order p of autoregressive terms. 101 00:06:34,100 --> 00:06:38,449 Then once we have a lot of models that we can play 102 00:06:38,449 --> 00:06:42,709 with then we will somehow have to choose one of those models. 103 00:06:42,709 --> 00:06:45,836 Right. And then what are going to be our criteria, 104 00:06:45,836 --> 00:06:47,329 we're gonna have more than one criteria. 105 00:06:47,329 --> 00:06:50,990 I know that you have seen already Akaike Information Criteria. 106 00:06:50,990 --> 00:06:54,050 So AIC is one of the things we're going to look at. 107 00:06:54,050 --> 00:06:58,019 We are trying to hope to get the least AIC. 108 00:06:58,019 --> 00:07:02,821 We also want to get the least sum of the squared errors, 109 00:07:02,821 --> 00:07:04,610 we are also going to look at that. 110 00:07:04,610 --> 00:07:11,115 And then I'm going to introduce one more measure in a way to select a model, 111 00:07:11,115 --> 00:07:17,620 and this measure is going to be basically Ljung-Box Q-statistics. 112 00:07:17,620 --> 00:07:21,125 So Ljung-Box Q-statistics, I'm going to talk about them next lecture. 113 00:07:21,125 --> 00:07:23,132 Basically by looking at these three, 114 00:07:23,132 --> 00:07:25,850 we'll try to select our model. 115 00:07:25,850 --> 00:07:27,649 Once we have our model, 116 00:07:27,649 --> 00:07:31,444 we're going to go through the estimation and try to fit the model 117 00:07:31,444 --> 00:07:35,600 to our time series. So what have you learned? 118 00:07:35,600 --> 00:07:38,055 We have learned how to describe autoregressive, 119 00:07:38,055 --> 00:07:40,085 integrated, moving average models. 120 00:07:40,085 --> 00:07:43,161 And we learned how to rewrite autoregressive, integrated, 121 00:07:43,161 --> 00:07:47,730 moving average models using backshift and difference operators.