Facial Action Units for Training Convolutional Neural Networks

This paper deals with the problem of training convolutional neural networks (CNNs) with facial action units (AUs). In particular, we focus on the imbalance problem of the training datasets for facial emotion classification. Since training a CNN with an imbalanced dataset tends to yield a learning bi...

Full description

Saved in:
Bibliographic Details
Published in:IEEE access 2019, Vol.7, p.77816-77824
Main Authors: Pham, Trinh Thi Doan, Won, Chee Sun
Format: Article
Language:English
Subjects:
Accuracy
Artificial neural networks
Classification
Convolutional neural network
Convolutional neural networks
data imbalance
data oversampling
Datasets
Emotions
Face
facial action units
facial emotion recognition
Gold
Image classification
Image retrieval
Neural networks
Oversampling
Support vector machines
Training
Training data
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper deals with the problem of training convolutional neural networks (CNNs) with facial action units (AUs). In particular, we focus on the imbalance problem of the training datasets for facial emotion classification. Since training a CNN with an imbalanced dataset tends to yield a learning bias toward the major classes and eventually leads to deterioration in the classification accuracy, it is required to increase the number of training images for the minority classes to have evenly distributed training images over all classes. However, it is difficult to find the images with a similar facial emotion for the oversampling. In this paper, we propose to use the AU features to retrieve an image with a similar emotion. The query selection from the minority class and the AU-based retrieval processes repeat until the numbers of training data over all classes are balanced. Also, to improve the classification accuracy, the AU features are fused with the CNN features to train a support vector machine (SVM) for final classification. The experiments have been conducted on three imbalanced facial image datasets, RAF-DB, FER2013, and ExpW. The results demonstrate that the CNNs trained with the AU features improve the classification accuracy by 3%-4%.
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2019.2921241