SETAR-Tree: a novel and accurate tree algorithm for global time series forecasting

• Published in: Machine learning 2023-07, Vol.112 (7), p.2555-2591
• Main Authors: Godahewa, Rakshitha, Webb, Geoffrey I., Schmidt, Daniel, Bergmeir, Christoph
• Format: Article
• Language: English
• Subjects: Accuracy; Algorithms; Artificial Intelligence; Autoregressive models; Computer Science; Control; Error analysis; Error reduction; Forecasting; Linearity; Machine Learning; Mechatronics; Natural Language Processing (NLP); Regression analysis; Regression models; Robotics; Simulation and Modeling; Special Issue of the ECML PKDD 2022 Journal Track; Statistical analysis; Time series
• ISSN: 0885-6125; 1573-0565
• DOI: 10.1007/s10994-023-06316-x

Threshold Autoregressive (TAR) models have been widely used by statisticians for non-linear time series forecasting during the past few decades, due to their simplicity and mathematical properties. On the other hand, in the forecasting community, general-purpose tree-based regression algorithms (forests, gradient-boosting) have become popular recently due to their ease of use and accuracy. In this paper, we explore the close connections between TAR models and regression trees. These enable us to use the rich methodology from the literature on TAR models to define a hierarchical TAR model as a regression tree that trains globally across series, which we call SETAR-Tree. In contrast to the general-purpose tree-based models that do not primarily focus on forecasting, and calculate averages at the leaf nodes, we introduce a new forecasting-specific tree algorithm that trains global Pooled Regression (PR) models in the leaves allowing the models to learn cross-series information and also uses some time-series-specific splitting and stopping procedures. The depth of the tree is controlled by conducting a statistical linearity test commonly employed in TAR models, as well as measuring the error reduction percentage at each node split. Thus, the proposed tree model requires minimal external hyperparameter tuning and provides competitive results under its default configuration. We also use this tree algorithm to develop a forest where the forecasts provided by a collection of diverse SETAR-Trees are combined during the forecasting process. In our evaluation on eight publicly available datasets, the proposed tree and forest models are able to achieve significantly higher accuracy than a set of state-of-the-art tree-based algorithms and forecasting benchmarks across four evaluation metrics.