Semi-Supervised Learning for Continuous Emotion Recognition Based on Metric Learning

• Published in: IEEE access 2020, Vol.8, p.113443-113455
• Main Authors: Choi, Dong Yoon, Song, Byung Cheol
• Format: Article
• Language: English
• Subjects: Annotations; arousal/valence domain; Artificial intelligence; CNN-LSTM; Datasets; deep metric learning; Emotion recognition; Emotions; Entropy; Feature extraction; Labels; Machine learning; Measurement; Object recognition; Semi-supervised learning; Semisupervised learning; Training
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2020.3003125

Emotion recognition is important for the interaction between human and artificial intelligence. Recently, the performance of facial image-based emotion recognition has been improved with deep learning's power. Nonetheless, huge data and label information for training are burdensome. In particular, annotating emotion labels in the continuous domain is very costly. Thus, we propose a novel semi-supervised learning that can not only reduce the annotation cost, but also improve emotion recognition performance by training with additional unlabeled data. The proposed method employs deep metric learning to improve feature embedding performance. Also, pseudo labels of unlabeled data are produced by analyzing inter-data distance in the feature space. Since pseudo labeling makes unlabeled data trainable, it increases overall performance. The experimental results show that the proposed method provides outstanding performance in the well-known MAHNOB-HCI dataset and the INHA dataset produced by our research team.