SVM-based feature selection methods for emotion recognition from multimodal data

• Published in: Journal on multimodal user interfaces 2017-03, Vol.11 (1), p.9-23
• Main Authors: Torres-Valencia, Cristian, Álvarez-López, Mauricio, Orozco-Gutiérrez, Álvaro
• Format: Article
• Language: English
• Subjects: Affective computing; Arousal; Classifiers; Computer Science; Discrimination; Electrocardiography; Electroencephalography; Emotion recognition; Emotions; Feature extraction; Feature selection; Galvanic skin response; Heart rate; Image Processing and Computer Vision; Original Paper; Pattern recognition; Pattern recognition systems; Physiology; Signal,Image and Speech Processing; Support vector machines; User Interfaces and Human Computer Interaction
• ISSN: 1783-7677; 1783-8738
• DOI: 10.1007/s12193-016-0222-y

Multimodal emotion recognition is an emerging field within affective computing that, by simultaneously using different physiological signals, looks for evaluating an emotional state. Physiological signals such as electroencephalogram (EEG), temperature and electrocardiogram (ECG), to name a few, have been used to assess emotions like happiness, sadness or anger, or to assess levels of arousal or valence. Research efforts in this field so far have mainly focused on building pattern recognition systems with an emphasis on feature extraction and classifier design. A different set of features is extracted over each type of physiological signal, and then all these sets of features are combined, and used to feed a particular classifier. An important stage of a pattern recognition system that has received less attention within this literature is the feature selection stage. Feature selection is particularly useful for uncovering the discriminant abilities of particular physiological signals. The main objective of this paper is to study the discriminant power of different features associated to several physiological signals used for multimodal emotion recognition. To this end, we apply recursive feature elimination and margin-maximizing feature elimination over two well known multimodal databases, namely, DEAP and MAHNOB-HCI. Results show that EEG-related features show the highest discrimination ability. For the arousal index, EEG features are accompanied by Galvanic skin response features in achieving the highest discrimination power, whereas for the valence index, EEG features are accompanied by the heart rate features in achieving the highest discrimination power.