Multi-step forecasting of multivariate time series using multi-attention collaborative network

• Published in: Expert systems with applications 2023-01, Vol.211, p.118516, Article 118516
• Main Authors: He, Xiaoyu, Shi, Suixiang, Geng, Xiulin, Yu, Jie, Xu, Lingyu
• Format: Article
• Language: English
• Subjects: Deep neural network; Multi-attention; Multi-step forecasting; Multivariate time series
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2022.118516

Multi-step forecasting of multivariate time series plays a critical role in many fields, such as disaster warning and financial analysis. While attention-based recurrent neural networks (RNNs) achieved encouraging performance, two limitations exist in current models: i) Existing approaches merely focus on variables’ interactions, and ignore the negative noise of non-predictive variables, ii) These methods cannot model the difference in the temporal importance of the target and the non-predictive series to prediction. To tackle these challenges, we propose a triangle structured Multi-Attention Collaborative Network (MACN), which includes a backbone network T-net with attention-based encoder–decoder framework, and an auxiliary hierarchical network NP-net. NP-net focuses on non-predictive variables, capturing the most relevant variables and temporal dependencies through the proposed variables-distillation attention network (VDN) and long short-term memory network (LSTM). T-net executes on target variable, and its encoder and decoder are both connected to NP-net, thereby using the output of NP-net to assist learning and decision-making. Specifically we design a knowledge-enhanced LSTM (KeLSTM) as the encoder and decoder of T-net. In the coding stage, KeLSTM refines the output of NP-net to strengthen the latent semantics of the target variable. In the decoding stage, KeLSTM captures subtle differences between the target and the non-predictive variables’ contribution to prediction, and improves model’s predictive ability by alleviating such conflicts. Experiments on three real-world datasets demonstrate that MACN outperforms different types of state-of-the-art methods. •A triangle structured multi-attention model for multivariate time series prediction.•Variables-distillation attention selects the most relevant non-predictive variables.•The KeLSTM uses non-predictive variables to enrich the target variable’s semantics.•The model distinguishes the effects of target and non-predictive variables on tasks.•MACN is competitive in multivariate time series multi-step prediction.