One thing I’ve found in my travels is that challenges surrounding forecasting in business often take precedence over statistical purity. Back in the day, I could devote special attention to each of a small number of forecasts I was responsible for, examining countless graphs, statistics, autocorrelation functions, etc. to choose a “best” model. That approach breaks down, however, when there are scores or even hundreds of separate series to forecast – as is often the case today. I now obsess on prediction quality over statistical purity, and generally prefer techniques that choose “optimal” solutions automatically based on performance criteria. I guess I’ve become a hardened algorithmist.
There’s no shortage of forecasting functionality available in R. Indeed, so significant is forecasting to R that the community supports a task view just to keep users apprised of current capabilities. It’s telling that this task view is maintained by time series expert and R “forecast” package author Rob Hyndman.
Several years ago, I fell in love with the forecast package, and now use it regularly in my work. There are plenty of goodies in forecast, aptly described by the author as “Methods and tools for displaying and analysing univariate time series forecasts, including exponential smoothing via state space models and automatic ARIMA modeling”.
Though I can’t show customer data, I can illustrate some of the thinking for the work, drawing on my trusty time series data set sourced from 14 years of daily births in Quebec, Canada. For the following analyses, I subset the 731 days between 1979-01-01 and 1980-12-31 as training data, testing different models to forecast the first 60 days of 1981. I then compare those predictions to the actuals I keep on the sideline.
Figure 1 provides several views of the training data. 1a is a basic time series plot detailing all 731 values. The data are pretty well behaved: the median number of births for all training data is 275, while the range is a narrow 165-366. 1b shows the mean number of births by day of the week for the training data; 1c does the same for month of year. These graphs clearly demonstrate day of week and month of year “seasons”. The relatively low numbers of Saturday and Sunday deliveries likely attest to planned weekday Cesarean Sections.
First up for the 60 day forecasting “competition” are basic linear regression models, with Figure 2 detailing the results. 2a uses a forecast package function to fit trend and the day of the week “season”. In 2b, I construct a linear model with day of the week and month as regressors. Once the models are fit to the training data, I invoke the forecast methods to predict births 60 days out.
In each graph, black indicates actual numbers while red the predictions based on the training set. Both linear models appear to do a pretty good job capturing the day of week pattern. The Mean Absolute Errors (MAEs), which measure the differences between actual and predicted values, are 16 and 15.13 respectively, each less than 6% of median daily births. I’d kill for performance like that with my current project, where the data are much more contrary.
Figure 3 shows the results of two variants of the computationally-intensive exponential smoothing, state space models. 3a demonstrates the ets function, which produces a “best” model among many candidates using the information criterion. 3b details predictions from a bats model – exponential smoothing with a bunch of extras. With a considerably smaller MAE and a much tighter graphical correspondence of actual to predicted data, bats is the winner in this comparison.
Autoregressive, Integrated, Moving Average (ARIMA) models have been a staple of the forecaster’s tool chest for over 40 years and are probably my first choice for predictive modeling. The problem is that “identifying” the lags, differences and seasons for ARIMA models is as much art as science. The forecast package provides a function, auto.arima, that can help automate those tasks, however, providing a “best” model according to the AIC criterion. Best is not very good in this case though. Figure 4 details the lackluster ARIMA predictions to the Quebec data. Had the prediction period been 30 days instead of 60, ARIMA would have shone.
One thing I’ve learned from lots of this work is that there’s not a single modeling technique that’s optimal for all forecasting scenarios. In fact, I often find that a model that kicks butt for one series falls short with another. And one that predicts 30 days accurately might not do so well for 60 days. Alas, modelers are forced to live with the predictive inconsistency.
My “gut” from experience over the last few years, though, suggests that basic regression, ARIMA and exponential smoothing models will handle most of the business forecasting problems I throw at them. The challenge is to determine the secret sauce of how best to deploy the disparate forecasts with those tools.