Subject independent facial expression recognition with robust face detection using a convolutional neural network

• Published in: Neural networks 2003-06, Vol.16 (5), p.555-559
• Main Authors: Matsugu, Masakazu, Mori, Katsuhiko, Mitari, Yusuke, Kaneda, Yuji
• Format: Article
• Language: English
• Subjects: Applied sciences; Convolutional neural network; Discrimination (Psychology) - physiology; Exact sciences and technology; Face detection; Facial Expression; Image processing; Information, signal and communications theory; Neural Networks (Computer); Pattern recognition; Recognition (Psychology) - physiology; Signal processing; Telecommunications and information theory
• ISSN: 0893-6080; 1879-2782
• DOI: 10.1016/S0893-6080(03)00115-1

Reliable detection of ordinary facial expressions (e.g. smile) despite the variability among individuals as well as face appearance is an important step toward the realization of perceptual user interface with autonomous perception of persons. We describe a rule-based algorithm for robust facial expression recognition combined with robust face detection using a convolutional neural network. In this study, we address the problem of subject independence as well as translation, rotation, and scale invariance in the recognition of facial expression. The result shows reliable detection of smiles with recognition rate of 97.6% for 5600 still images of more than 10 subjects. The proposed algorithm demonstrated the ability to discriminate smiling from talking based on the saliency score obtained from voting visual cues. To the best of our knowledge, it is the first facial expression recognition model with the property of subject independence combined with robustness to variability in facial appearance.