Convolutional Transformer: An Enhanced Attention Mechanism Architecture for Remaining Useful Life Estimation of Bearings

Nowadays, deep learning (DL) methods for prognostic and health management (PHM) have vastly broadened the scope of applications in this field. Numerous approaches based on deep neural networks have been presented and applied to the remaining useful life (RUL) estimation of bearings. However, few of...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on instrumentation and measurement 2022, Vol.71, p.1-10
Main Authors: Ding, Yifei, Jia, Minping
Format: Article
Language:English
Subjects:
Ablation
Artificial neural networks
Attention mechanism
bearings
Comparative studies
Computer architecture
Convolution
convolutional neural network (CNN)
Deep learning
Feature extraction
Machine learning
Modelling
Prognostics and health management
remaining useful life (RUL)
Task analysis
transformer
Transformers
Useful life
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Nowadays, deep learning (DL) methods for prognostic and health management (PHM) have vastly broadened the scope of applications in this field. Numerous approaches based on deep neural networks have been presented and applied to the remaining useful life (RUL) estimation of bearings. However, few of these methods are yet fully competent for the task of extracting degradation-related information from raw signals both locally and globally. To fill this research gap, we proposed a novel convolutional Transformer (CoT) that combines the global context capturing of attention mechanism with the local dependencies modeling of convolutional operation. Specifically, we designed a multiscale convolutional (MSC) module with Swish activation for Transformer architecture to embed local feature learning into global sequence modeling. Our CoT fuses intratoken convolution and intertoken self-attention operations to enable the simultaneous extraction of local dependencies and global interactions from the raw temporal signal into a trainable class token. Then, an end-to-end RUL estimation framework based on CoT is presented, which provides a mapping from raw vibration signals to estimated RULs. Finally, comprehensive case studies, including comparative studies and ablation experiments, fully validate the effectiveness and advancements of our CoT-based RUL estimation method.
ISSN:0018-9456
1557-9662
DOI:10.1109/TIM.2022.3181933