M ^ D-VAE: Self-Supervised Probabilistic Temporal- Spatial Latent Representation Learning for Unsupervised Industrial Operational Applications Under Missing Value Interference

• Published in: IEEE transaction on neural networks and learning systems 2024-10, p.1-14
• Main Authors: Dai, Qingyang, Zhao, Chunhui, Huang, Biao
• Format: Article
• Language: English
• Subjects: Data mining; Data models; Dynamic data modeling; Feature extraction; Hidden Markov models; Imputation; industrial process monitoring; Interference; missing data; Optimization; Probabilistic logic; Representation learning; Self-supervised learning; variational autoencoders (VAEs)
• ISSN: 2162-237X; 2162-2388
• DOI: 10.1109/TNNLS.2024.3477968

Due to sensor malfunctions and data transmission corruptions, the industrial process data collected commonly contain missing values. It poses a significant challenge for data-driven approaches in aggregating temporal-spatial correlations that reflect dependencies across both variables and times, which makes it difficult to directly carry out downstream industrial operational applications. In this study, a self-supervised representation learning model is proposed to extract probabilistic temporal-spatial latent variables (LVs) from sequential data under missing value interference. The extracted LVs can be utilized for typical industrial operational applications through a unified framework. First, a novel deep dynamic probabilistic latent variable model, named Markov dynamic variational autoencoder (MD-VAE), is proposed to explicitly model the temporal-spatial dependencies between LVs. The latent posteriors are Bayesian smoothed by global sequence information for effective variational inference (VI). Second, a self-supervised learning approach, termed masked MD-VAE (), is proposed to address the challenge of directly extracting temporal-spatial LVs under missing value interference. Controllable constraints with practical interpretations are introduced to balance the latent bottleneck capacity with reconstruction accuracy during model optimization. A unified framework is proposed to utilize the latent representations for typical industrial downstream tasks. Case studies conducted on a real-world multiphase flow process demonstrate the superiority of in unsupervised industrial operational applications including missing value imputation and dynamic process monitoring under missing value interference.