Fully Automatic Recognition of the Temporal Phases of Facial Actions

Past work on automatic analysis of facial expressions has focused mostly on detecting prototypic expressions of basic emotions like happiness and anger. The method proposed here enables the detection of a much larger range of facial behavior by recognizing facial muscle actions [action units (AUs)]...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on cybernetics 2012-02, Vol.42 (1), p.28-43
Main Authors: Valstar, M. F., Pantic, M.
Format: Article
Language:English
Subjects:
Artificial Intelligence
Cybernetics
Detectors
Emotion recognition
Emotions
Face
Face recognition
Facial
Facial Expression
Facial expression analysis
Filtering
GentleBoost
Gold
Hidden Markov models
Humans
Image Interpretation, Computer-Assisted - methods
Image sequences
particle filtering
Pattern Recognition, Automated - methods
Photography - methods
Recognition
spatiotemporal facial behavior analysis
Subtraction Technique
support vector machine (SVM)
Temporal logic
Video Recording - methods
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Past work on automatic analysis of facial expressions has focused mostly on detecting prototypic expressions of basic emotions like happiness and anger. The method proposed here enables the detection of a much larger range of facial behavior by recognizing facial muscle actions [action units (AUs)] that compound expressions. AUs are agnostic, leaving the inference about conveyed intent to higher order decision making (e.g., emotion recognition). The proposed fully automatic method not only allows the recognition of 22 AUs but also explicitly models their temporal characteristics (i.e., sequences of temporal segments: neutral, onset, apex, and offset). To do so, it uses a facial point detector based on Gabor-feature-based boosted classifiers to automatically localize 20 facial fiducial points. These points are tracked through a sequence of images using a method called particle filtering with factorized likelihoods. To encode AUs and their temporal activation models based on the tracking data, it applies a combination of GentleBoost, support vector machines, and hidden Markov models. We attain an average AU recognition rate of 95.3% when tested on a benchmark set of deliberately displayed facial expressions and 72% when tested on spontaneous expressions.
ISSN:1083-4419
2168-2267
1941-0492
2168-2275
DOI:10.1109/TSMCB.2011.2163710