Improved EEG-based emotion recognition through information enhancement in connectivity feature map

• Published in: Scientific reports 2023-08, Vol.13 (1), p.13804-17, Article 13804
• Main Authors: Akhand, M. A. H., Maria, Mahfuza Akter, Kamal, Md Abdus Samad, Murase, Kazuyuki
• Format: Article
• Language: English
• Subjects: 631/378/1457; 639/166/985; Benchmarking; Brain; Computational neuroscience; Correlation coefficient; EEG; Electroencephalography; Emotions; Entropy; Frequency dependence; Humanities and Social Sciences; Humans; Learning algorithms; Machine learning; multidisciplinary; Neural networks; Recognition, Psychology; Science; Science (multidisciplinary)
• ISSN: 2045-2322; 2045-2322
• DOI: 10.1038/s41598-023-40786-2

Electroencephalography (EEG), despite its inherited complexity, is a preferable brain signal for automatic human emotion recognition (ER), which is a challenging machine learning task with emerging applications. In any automatic ER, machine learning (ML) models classify emotions using the extracted features from the EEG signals, and therefore, such feature extraction is a crucial part of ER process. Recently, EEG channel connectivity features have been widely used in ER, where Pearson correlation coefficient (PCC), mutual information (MI), phase-locking value (PLV), and transfer entropy (TE) are well-known methods for connectivity feature map (CFM) construction. CFMs are typically formed in a two-dimensional configuration using the signals from two EEG channels, and such two-dimensional CFMs are usually symmetric and hold redundant information. This study proposes the construction of a more informative CFM that can lead to better ER. Specifically, the proposed innovative technique intelligently combines CFMs’ measures of two different individual methods, and its outcomes are more informative as a fused CFM. Such CFM fusion does not incur additional computational costs in training the ML model. In this study, fused CFMs are constructed by combining every pair of methods from PCC, PLV, MI, and TE; and the resulting fused CFMs PCC + PLV, PCC + MI, PCC + TE, PLV + MI, PLV + TE, and MI + TE are used to classify emotion by convolutional neural network. Rigorous experiments on the DEAP benchmark EEG dataset show that the proposed CFMs deliver better ER performances than CFM with a single connectivity method (e.g., PCC). At a glance, PLV + MI-based ER is shown to be the most promising one as it outperforms the other methods.