Dynamic Co-Attention Networks for multi-horizon forecasting in multivariate time series

• Published in: Future generation computer systems 2022-10, Vol.135, p.72-84
• Main Authors: He, Xiaoyu, Shi, Suixiang, Geng, Xiulin, Xu, Lingyu
• Format: Article
• Language: English
• Subjects: Co-attention; Deep neural network; Information fusion; Interpretability; Multi-horizon forecasting; Multivariate time series
• ISSN: 0167-739X; 1872-7115
• DOI: 10.1016/j.future.2022.04.029

Although attention-based encoder–decoder models achieve encouraging performance in multivariate time series multi-horizon forecasting, two key limitations exist in current methods: (i) These methods merely focus on the spatial or temporal correlations among series, and ignore harmful interactions between non-predictive and target variables. (ii) Time series forecasting benefits significantly from knowledge of the recent past, but existing approaches only consider the encoder and neglect the previous decoder units. To tackle these challenges, we propose a novel prediction framework, termed Dynamic Co-Attention Networks (DCAN). For the first issue, we propose a two-stage variables embedding network as DCAN’s underlying structure to better capture the potential semantics that integrates target and non-predictive variables. The first stage selects the most important non-predictive variables and generates a spatial vector, and the second stage trades off how much information the potential semantics consider from the target variable and spatial vector. To alleviate the second issue, we design a dynamic attention synergetic network (DASN) as the superstructure of the proposed model, including a contextual attention and a filtration gate. A contextual attention expands the observation horizon for current prediction by continuously retrieving previous decoder states, and a filtration gate module selectively incorporates previous decoder information into the current decoder unit, thus improving the credibility and accuracy of the model decision-making. Experiments on three real-life datasets demonstrate that DCAN outperforms different types of state-of-the-art baselines. •An multi-attention network for multivariate time series multi-horizon prediction.•The underlying structure TSVEN mitigates negative interactions among input variables.•The superstructure DASN provides extra historical information for the current forecast.•The proposed VDN module provides interpretability for predictions.