A Multimodal Dataset for Mixed Emotion Recognition

Mixed emotions have attracted increasing interest recently, but existing datasets rarely focus on mixed emotion recognition from multimodal signals, hindering the affective computing of mixed emotions. On this basis, we present a multimodal dataset with four kinds of signals recorded while watching...

Full description

Saved in:
Bibliographic Details
Published in:Scientific data 2024-08, Vol.11 (1), p.847-14, Article 847
Main Authors: Yang, Pei, Liu, Niqi, Liu, Xinge, Shu, Yezhi, Ji, Wenqi, Ren, Ziqi, Sheng, Jenny, Yu, Minjing, Yi, Ran, Zhang, Dan, Liu, Yong-Jin
Format: Article
Language:English
Subjects:
639/705/1046
639/705/117
Classification
Data Descriptor
Datasets
Electroencephalography
Emotions
Facial Expression
Humanities and Social Sciences
Humans
multidisciplinary
Science
Science (multidisciplinary)
Self-assessment
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Mixed emotions have attracted increasing interest recently, but existing datasets rarely focus on mixed emotion recognition from multimodal signals, hindering the affective computing of mixed emotions. On this basis, we present a multimodal dataset with four kinds of signals recorded while watching mixed and non-mixed emotion videos. To ensure effective emotion induction, we first implemented a rule-based video filtering step to select the videos that could elicit stronger positive, negative, and mixed emotions. Then, an experiment with 80 participants was conducted, in which the data of EEG, GSR, PPG, and frontal face videos were recorded while they watched the selected video clips. We also recorded the subjective emotional rating on PANAS, VAD, and amusement-disgust dimensions. In total, the dataset consists of multimodal signal data and self-assessment data from 73 participants. We also present technical validations for emotion induction and mixed emotion classification from physiological signals and face videos. The average accuracy of the 3-class classification (i.e., positive, negative, and mixed) can reach 80.96% when using SVM and features from all modalities, which indicates the possibility of identifying mixed emotional states.
ISSN:2052-4463
2052-4463
DOI:10.1038/s41597-024-03676-4