Variational Attention-Based Interpretable Transformer Network for Rotary Machine Fault Diagnosis

Deep learning technology provides a promising approach for rotary machine fault diagnosis (RMFD), where vibration signals are commonly utilized as input of a deep network model to reveal the internal state of machinery. However, most existing methods fail to mine association relationships within sig...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transaction on neural networks and learning systems 2024-05, Vol.35 (5), p.6180-6193
Main Authors: Li, Yasong, Zhou, Zheng, Sun, Chuang, Chen, Xuefeng, Yan, Ruqiang
Format: Article
Language:English
Subjects:
Analytical models
Approximation
Artificial neural networks
Bevel gears
Convolutional neural networks
Deep learning
Dirichlet problem
Fault diagnosis
Feature extraction
Heating systems
Interpretability
Machine learning
Neural networks
rotary machine fault diagnosis (RMFD)
Rotary machines
transformer
Transformers
variational attention
Vibration
Vibrations
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Deep learning technology provides a promising approach for rotary machine fault diagnosis (RMFD), where vibration signals are commonly utilized as input of a deep network model to reveal the internal state of machinery. However, most existing methods fail to mine association relationships within signals. Unlike deep neural networks, transformer networks are capable of capturing association relationships through the global self-attention mechanism to enhance feature representations from vibration signals. Despite this, transformer networks cannot explicitly establish the causal association between signal patterns and fault types, resulting in poor interpretability. To tackle these problems, an interpretable deep learning model named the variational attention-based transformer network (VATN) is proposed for RMFD. VATN is improved from transformer encoder to mine the association relationships within signals. To embed the prior knowledge of the fault type, which can be recognized based on several key features of vibration signals, a sparse constraint is designed for attention weights. Variational inference is employed to force attention weights to samples from Dirichlet distributions, and Laplace approximation is applied to realize reparameterization. Finally, two experimental studies conducted on bevel gear and bearing datasets demonstrate the effectiveness of VATN to other comparison methods, and the heat map of attention weights illustrates the causal association between fault types and signal patterns.
ISSN:2162-237X
2162-2388
DOI:10.1109/TNNLS.2022.3202234