Automatic detection of artifacts and improved classification models for emotional activity detection from multimodal physiological data

• Published in: Journal of intelligent & fuzzy systems 2023-11, Vol.45 (5), p.8915-8929
• Main Authors: Prabhakaran, Sudarsan, Ayyamperumal, Niranjil Kumar
• Format: Article
• Language: English
• Subjects: Algorithms; Artificial neural networks; Blood volume; Classification; Classifiers; Electrocardiography; Emotions; Machine learning; Neural networks; Physiology; Respiration; Signal classification
• ISSN: 1064-1246; 1875-8967
• DOI: 10.3233/JIFS-232662

This manuscript proposes an automated artifacts detection and multimodal classification system for human emotion analysis from human physiological signals. First, multimodal physiological data, including the Electrodermal Activity (EDA), electrocardiogram (ECG), Blood Volume Pulse (BVP) and respiration rate signals are collected. Second, a Modified Compressed Sensing-based Decomposition (MCSD) is used to extract the informative Skin Conductance Response (SCR) events of the EDA signal. Third, raw features (edge and sharp variations), statistical and wavelet coefficient features of EDA, ECG, BVP, respiration and SCR signals are obtained. Fourth, the extracted raw features, statistical and wavelet coefficient features from all physiological signals are fed into the parallel Deep Convolutional Neural Network (DCNN) to reduce the dimensionality of feature space by removing artifacts. Fifth, the fused artifact-free feature vector is obtained for neutral, stress and pleasure emotion classes. Sixth, an artifact-free feature vector is used to train the Random Forest Deep Neural Network (RFDNN) classifier. Then, a trained RFDNN classifier is applied to classify the test signals into different emotion classes. Thus, leveraging the strengths of both RF and DNN algorithms, more comprehensive feature learning using multimodal psychological data is achieved, resulting in robust and accurate classification of human emotional activities. Finally, an extensive experiment using the Wearable Stress and Affect Detection (WESAD) dataset shows that the proposed system outperforms other existing human emotion classification systems using physiological data.