A learnable continuous wavelet-based multi-branch attentive convolutional neural network for spatio–spectral–temporal EEG signal decoding

• Published in: Expert systems with applications 2024-10, Vol.251, p.123975, Article 123975
• Main Authors: Kim, Jun-Mo, Heo, Keun-Soo, Shin, Dong-Hee, Nam, Hyeonyeong, Won, Dong-Ok, Jeong, Ji-Hoon, Kam, Tae-Eui
• Format: Article
• Language: English
• Subjects: Brain–computer interface (BCI); Convolutional neural network (CNN); Electroencephalography (EEG); Motor imagery; Wavelet transform
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2024.123975

Deep learning methods, particularly convolutional neural networks (CNNs), have advanced the field of motor imagery (MI) by effectively extracting spatio-spectral–temporal (SST) features from electroencephalography (EEG) signals. Image-based CNNs using time–frequency analysis such as wavelet transform have outperformed signal-based CNNs in MI-EEG classification. However, the spatial characteristics of EEG signals can be neglected due to the high dimensionality of spectral–temporal feature space. Moreover, the performance hinges on whether the predefined transformation can capture the class-discriminative spectral–temporal representations of EEG signals well. Despite attempts to alleviate these challenges by selecting EEG electrodes or restricting spectral–temporal feature space, large inter-subject variabilities have impeded successful MI-EEG decoding. In this paper, we propose a learnable continuous wavelet-based multi-branch attentive CNN framework for decoding MI-EEG signals. Our method is designed to automatically generate class-discriminative SST representations of EEG signals within subject-specific sub-bands by utilizing learnable wavelet-based convolutions. Additionally, it employs multi-branch attentive CNNs for the effective and efficient extraction of local and global SST features. Our comprehensive evaluation on two public datasets demonstrates that the proposed method significantly outperformed state-of-the-art methods. We also conduct an ablation study under various configuration settings within the proposed framework to demonstrate the effectiveness of our method. The proposed method provides a powerful and innovative approach to MI-EEG signal decoding, with implications for personalized EEG decoding. •Motor Imagery-EEG decoding plays a critical role in neurorehabilitation and BCI.•We introduce a novel method for spatio-spectral–temporal motor imagery-EEG decoding.•Our method transforms EEG into discriminative SST representations automatically.•Our multi-branch attentive CNN learns local/global SST features simultaneously.•We demonstrate its effectiveness on public datasets by comparing with SOTA methods.