Ternary-task convolutional bidirectional neural turing machine for assessment of EEG-based cognitive workload

• Published in: Biomedical signal processing and control 2020-03, Vol.57, p.101745, Article 101745
• Main Authors: Qiao, Weizheng, Bi, Xiaojun
• Format: Article
• Language: English
• Subjects: Cognitive workload classification; Convolutional bidirection neural turing machine; Deep learning; EEG; Ternary-task learning
• ISSN: 1746-8094; 1746-8108
• DOI: 10.1016/j.bspc.2019.101745

•The combination of CNN and BNTM can better preserve spatial, spectral, and temporal information of multi-channel EEG time-series.•Our improvement on NTM by adding bidirectional mechanism helps to improve performance in modeling the context information.•Ternary-task learning regularization can effectively overcome overfitting based on identification and verification task. Cognitive workload plays a crucial role in the observation of mental activity which has great significance in brain-computer interfaces (BCI), cognitive neuroscience and biomedical fields. The estimation of cognitive state based on the classification of the electroencephalograph (EEG) is a hot issue received more and more attention. So far, a variety of Deep Learning models have been raised, which has yielded improvements in feature extraction and classification. However, the existing models reveal shortcomings in processing spatial, spectral, and temporal features of the EEG. In this paper, we propose a deep hybrid Network called Ternary-task Convolutional Bidirectional Neural Turing Machine (TT-CBNTM) to perform cognitive state assessment. First, TT-CBNTM consists of Convolutional Neural Network (CNN) and Bidirectional Neural Turing Machine (BNTM), of which CNN is applied to preserve the spatial and spectral representations of EEG, while the BNTM is applied to learn temporal representations from features which are extracted from the CNN. Second, we propose a new strategy called ternary-task regularization framework to induce the overfitting on the EEG database. The main task is to assess EEG-based cognitive workload through classification of EEG signals. The auxiliary tasks is identification and verification, through which we can increase the inter-class variations and reduce the intra-class variations, further lead to a better result of the EEG-based cognitive workload classification. The classication accuracy of our TT-CBNTM is 96.3 %, yielding 5 % improvement over the state-of-the-art models. This demonstrates the significant effectiveness of our approach which can be applied successfully to cognitive monitoring systems.