Semi-supervised LSTM with historical feature fusion attention for temporal sequence dynamic modeling in industrial processes

• Published in: Engineering applications of artificial intelligence 2023-01, Vol.117, p.105547, Article 105547
• Main Authors: Tang, Yiyin, Wang, Yalin, Liu, Chenliang, Yuan, Xiaofeng, Wang, Kai, Yang, Chunhua
• Format: Article
• Language: English
• Subjects: Deep learning; Historical feature fusion attention (HFFA); Industrial processes; Semi-supervised long short-term memory (SSLSTM); Temporal sequence dynamic modeling
• ISSN: 0952-1976; 1873-6769
• DOI: 10.1016/j.engappai.2022.105547

In modern industrial processes, the data-driven soft sensor technology has been widely used for the prediction of key quality variables. Due to the important of dynamics and nonlinearity in industrial process data, deep learning models like long short-term memory (LSTM) network are well suited for temporal sequence dynamic modeling due to their excellent long-term memory function and feature extraction capability. Furthermore, industrial processes generate a large amount of process data with irregular sampling frequencies. However, traditional LSTM cannot fully utilize the process data with irregular sampling frequency and the guidance value of historical data samples for feature learning. To address these issues, a novel semi-supervised LSTM with history feature fusion attention (HFFA-SSLSTM) model is proposed in this paper. First, the semi-supervised learning strategy is implemented in LSTM to fully utilize the unlabeled data and mine the temporal sequence features of labeled samples and unlabeled samples with irregular sampling frequencies. Then, a novel historical feature fusion attention (HFFA) mechanism is developed, which utilizes historical hidden features to learn attention scores for obtaining weighted historical information-related features. Finally, the extracted features are combined to form the soft sensor model to perform time series prediction tasks for key quality variables in industrial processes. The experimental results on the actual industrial hydrocracking data set demonstrate the effectiveness of the proposed HFFA-SSLSTM model and its possibility of applicating in real industrial processes.