Prophecy™ Statistical forecasting engine
Dual forecasting engines
Prophecy V11.0 provides fully transparent, tightly coupled integration with R , the open source statistics and data science solution. This allows Prophecy to generate statistical forecasts using best-of-class R forecasting models, without the need for R or specialist statistical forecasing knowledge. Prophecy's advanced R forecasting is described on this page.
Previously, Prophecy always had its own built-in statistical forecasting engine, providing core time-series modelling without the need for statistical knowledge. The core engine contains the standard, most widely used time series forecasting algorithms but does not offer the same range of statistical and machine learning forecasting models are the R-integrated option..
Learn more about the original, Prophecy core statistical forecasting engine.
Run state-of-the art automatic forecasting algorithms, developed by renowned academic statisticians in R, from within Prophecy. (Scroll down the page for the current list of statistical methods that Prophecy supports.)
Prophecy offers a completely 'Automatic' mode that requires minimal statistics or R knowledge. You can choose methods or use a 'tournament of statistics' to automatically choose the best method for each product/customer combination. You can preview the resulting forecasts, both in graphs and spreadsheet-like views, before choosing, on an item by item basis, which forecasts to commit to Prophecy.
'Automatic' mode hides the complexity of R completely. It simply requires the user to select from one or more built-in R forecasting templates (models) available from Prophecy. It is ideal for the user who just needs to generate sensible forecasts quickly and with minimal statistical knowledge.
For users familiar with R there is 'R Expert' mode, which provides 'to the metal' access to the full R feature-set.
'R Expert' mode allows skilled R and statistical users to utilise R's full analytical, graphical and forecasting capabilities. It breaks the process into two steps. First, exporting data from Prophecy into R (RStudio). Second, bringing the forecasts back into Prophecy once they have been generated in R / RStudio. The bit in the middle - generating the forecasts - is done by the user using RStudio and the R language.
The following sections explain in more detail:
'Automatic' forecasting mode
The R forecasting models transparently automate parameter selection and generate sensible forecasts with no
user-intervention required. The Prophecy user simply selects one or more of the ‘R templates’ provided, specifies which products and customers to apply
them to, and clicks ‘Run’.
Prophecy then runs R as a background, invisible process to generate the forecasts. The user can then preview the resulting forecasts in a multi-dimensional grid view (similar to a Prophecy report) or with graphs depicting the R forecast versus history and any existing forecast in Prophecy.
The final stage in the ‘Automatic’ process is to apply the R forecasts back to the Prophecy database by clicking the 'Apply!' button.
In summary, ‘Automatic’ mode requires zero knowledge of R or
statistics. Use it to quickly get you as good a set of statistical forecasts as
today’s state of the art time series forecasting allows.
'R Expert' forecasting mode
99% of users will use Prophecy’s ‘Automatic’ mode and get great results,
because the built-in automatic forecasting routines in R are comparable with the best commercial statistical forecasting engines.
However, 'R Expert' mode is also available, to allow those users with statistics and R skills to access the full R feature-set from Prophecy.
Use Prophecy's ‘R Expert’ mode to export history from Prophecy into R for analysis, transformation, graphing, presentation and forecasting. Use the RStudio environment for 'to-the-metal- access to R and the sales data. Once you’ve generated forecasts in RStudio, use Prophecy’s ‘R Import’ button to read them back into Prophecy.
'Out of the box' forecasting templates
Prophecy comes with pre-built templates to use the following R forecast algorithms in both Automatic and Expert modes.
All the templates, where appropriate, provide the option to automatically clean up history without manual intervention - i.e. to replace outliers and zeros in the history with interpolated estimates. This behaviour can be removed if required, though for automatic forecasting it is not generally recommended.
The following statistical forecasting methods are provided 'out of the box':
Returns best ARIMA model according to either AIC, AICc or BIC value. The function conducts a search over possible models within the default order constraints.
Exponential smoothing state space model with Box-Cox transformation, ARMA errors, Trend and Seasonal components. As described in De Livera, Hyndman & Snyder (2011).
Based on Croston's (1972) method for intermittent demand forecasting, also described in Shenstone and Hyndman (2005). Croston's method involves using simple exponential smoothing (SES) on the non-zero elements of the time series and a separate application of SES to the times between non-zero elements of the time series.
Exponential smoothing state space model.
Parameters are determined by minimizing the squared prediction error.
Machine Learning - XGBoost
XGBoost is a machine learning (ML) library which implements the gradient boosting decision tree algorithm. This algorithm goes by lots of different names such as gradient boosting, multiple additive regression trees, stochastic gradient boosting or gradient boosting machines. Boosting is an ensemble technique where new models are added to correct the errors made by existing models. Models are added sequentially until no further improvements can be made. Gradient boosting is an approach where new models are created that predict the residuals or errors of prior models and then added together to make the final prediction. It is called gradient boosting because it uses a gradient descent algorithm to minimize the loss when adding new models. This approach supports both regression and classification predictive modeling problems. XGBoost has proven very successful in machine learning competitions and also executes quickly in R.
- Neural Network (Auto Regressive)
Feed-forward neural networks with a single hidden layer and lagged inputs for forecasting univariate time series.
Prophet is a procedure for forecasting time series data based on an additive model where non-linear trends are fit with yearly, weekly, and daily seasonality, plus holiday effects. It works best with time series that have strong seasonal effects and several seasons of historical data. Prophet is robust to missing data and shifts in the trend, and typically handles outliers well. Prophet is open source software released by Facebook's Core Data Science team.
Seasonal Decomposition of Time Series by Loess.
Trigonometric Seasonal, Box-Cox Transformation, ARMA residuals, Trend and Seasonality
- Multi-model Average
Averages the forecasts from Arima, BATS, Croston, ETS, HoltWinters. As suggested by https://otexts.com/fpp2/combinations.html.
- Multi-model Tournament
Runs a tournament of methods (Arima, BATS, Croston, ETS, HoltWinters) and chooses the method with the lowest MAPE, as calculated by the accuracy() function of the forecast library.