Neural network ensemble operators for time series forecasting

• Published in: Expert systems with applications 2014-07, Vol.41 (9), p.4235-4244
• Main Authors: Kourentzes, Nikolaos, Barrow, Devon K., Crone, Sven F.
• Format: Article
• Language: English
• Subjects: Applied sciences; Artificial intelligence; Automation; Combination; Computer science; control theory; systems; Connectionism. Neural networks; Density; Deviation; Ensembles; Exact sciences and technology; Forecasting; Inference from stochastic processes; time series analysis; Kernel density estimation; Mathematical models; Mathematics; Mean; Median; Mode estimation; Neural networks; Nonparametric inference; Operators; Probability and statistics; Sampling theory, sample surveys; Sciences and techniques of general use; Statistics; Time series
• ISSN: 0957-4174; 1873-6793; 1873-6793
• DOI: 10.1016/j.eswa.2013.12.011

•Different combination operators are evaluated for neural network ensembles.•The proposed mode operator is insensitive to outliers and asymmetric distributions.•The mode is found to provide the most accurate forecasts, followed by the median.•The commonly used mean operator ranks last requiring very large ensembles. The combination of forecasts resulting from an ensemble of neural networks has been shown to outperform the use of a single “best” network model. This is supported by an extensive body of literature, which shows that combining generally leads to improvements in forecasting accuracy and robustness, and that using the mean operator often outperforms more complex methods of combining forecasts. This paper proposes a mode ensemble operator based on kernel density estimation, which unlike the mean operator is insensitive to outliers and deviations from normality, and unlike the median operator does not require symmetric distributions. The three operators are compared empirically and the proposed mode ensemble operator is found to produce the most accurate forecasts, followed by the median, while the mean has relatively poor performance. The findings suggest that the mode operator should be considered as an alternative to the mean and median operators in forecasting applications. Experiments indicate that mode ensembles are useful in automating neural network models across a large number of time series, overcoming issues of uncertainty associated with data sampling, the stochasticity of neural network training, and the distribution of the forecasts.