Emotion Recognition from Physiological Signals Collected with a Wrist Device and Emotional Recall

• Published in: Bioengineering (Basel) 2023-11, Vol.10 (11), p.1308
• Main Authors: Mattern, Enni, Jackson, Roxanne R, Doshmanziari, Roya, Dewitte, Marieke, Varagnolo, Damiano, Knorn, Steffi
• Format: Article
• Language: English
• Subjects: Algorithms; Bioengineering; Blood pressure; Blood volume; Classification; Data collection; Deep learning; Electrocardiography; Electroencephalography; Emotion recognition; Emotions; Engineering; Feedback; Heart rate; Machine learning; Medical research; medical wristband; Memory tasks; Methods; Nervous system; Physical therapy; Physiological aspects; physiological signals; Physiology; Recall; Segments; Sensors; Skin temperature; Supervised learning; Support vector machines; Technology assessment; Vagina; Wrist
• ISSN: 2306-5354; 2306-5354
• DOI: 10.3390/bioengineering10111308

Implementing affective engineering in real-life applications requires the ability to effectively recognize emotions using physiological measurements. Despite being a widely researched topic, there seems to be a lack of systems that translate results from data collected in a laboratory setting to higher technology readiness levels. In this paper, we delve into the feasibility of emotion recognition beyond controlled laboratory environments. For this reason, we create a minimally-invasive experimental setup by combining emotional recall via autobiographical emotion memory tasks with a user-friendly Empatica wristband measuring blood volume pressure, electrodermal activity, skin temperature, and acceleration. We employ standard practices of feature-based supervised learning and specifically use support vector machines to explore subject dependency through various segmentation methods. We collected data from 45 participants. After preprocessing, using a data set of 134 segments from 40 participants, the accuracy of the classifier after 10-fold cross-validation was barely better than random guessing (36% for four emotions). However, when extracting multiple segments from each emotion task per participant using 10-fold cross-validation (i.e., including subject-dependent data in the training set), the classification rate increased to up to 75% for four emotions but was still as low as 32% for leave-one-subject-out cross-validation (i.e., subject-independent training). We conclude that highly subject-dependent issues might pose emotion recognition.