Deep Learning-Based Wearable Ear-EEG Emotion Recognition System With Superlets-Based Signal-to-Image Conversion Framework

This study presents a novel approach for affective computing through deep learning-assisted electroencephalography (EEG) analysis aimed at enhancing the detection, processing, interpretation, and emulation of human emotions. The proposed methodology introduces an Ear-EEG emotion recognition (EEER) s...

Full description

Saved in:
Bibliographic Details
Published in:IEEE sensors journal 2024-04, Vol.24 (7), p.11946-11958
Main Authors: Mai, Ngoc-Dau, Nguyen, Ha-Trung, Chung, Wan-Young
Format: Article
Language:English
Subjects:
Affective computing
Artificial intelligence
Artificial neural networks
Brain modeling
Brain–computer interface
Datasets
Deep learning
Ear
Electroencephalography
electroencephalography (EEG)
Emotion recognition
Emotional factors
Emotions
Internet of Things
Internet of Things (IoT)
Machine learning
Medical imaging
Remote monitoring
Scalp
Sensors
small-size dataset
Time-frequency analysis
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This study presents a novel approach for affective computing through deep learning-assisted electroencephalography (EEG) analysis aimed at enhancing the detection, processing, interpretation, and emulation of human emotions. The proposed methodology introduces an Ear-EEG emotion recognition (EEER) system, leveraging a deep neural network and integrating the Internet of Things (IoT) capabilities. The key contributions of this study include 1) a comprehensive design framework encompassing both hardware and software aspects of the Ear-EEG system, as well as 2) a signals-to-three-channel image conversion method employing time-frequency super-resolution with superlets as inputs for deep learning models. Moreover, 3) a modified vision transformer (ViT) architecture is introduced, incorporating shifted patch tokenization (SPT) and locality self-attention (LSA) techniques, addressing challenges associated with limited and imbalanced small datasets, as well as the lack of locality inductive bias for emotion recognition. Additionally, 4) an IoT-assisted EEER platform is proposed, enabling remote monitoring and management. Experimental results indicate that the trained ViT model with SPT and LSA surpasses recent models in terms of performance on untrained datasets, achieving an average accuracy of 92.39%. The findings highlight the efficacy of the proposed EEER system in accurately detecting emotional states, encompassing positive and negative affective states. The integration of artificial intelligence and IoT-based healthcare platforms represents a significant advancement in the development of medical assistant tools with broad implications for future applications.
ISSN:1530-437X
1558-1748
DOI:10.1109/JSEN.2024.3369062