Deep, Landmark-Free FAME: Face Alignment, Modeling, and Expression Estimation

We present a novel method for modeling 3D face shape, viewpoint, and expression from a single, unconstrained photo. Our method uses three deep convolutional neural networks to estimate each of these components separately. Importantly, unlike others, our method does not use facial landmark detection...

Full description

Saved in:
Bibliographic Details
Published in:International journal of computer vision 2019-06, Vol.127 (6-7), p.930-956
Main Authors: Chang, Feng-Ju, Tran, Anh Tuan, Hassner, Tal, Masi, Iacopo, Nevatia, Ram, Medioni, Gérard
Format: Article
Language:English
Subjects:
Accuracy
Alignment
Artificial Intelligence
Artificial neural networks
Benchmarks
Computer Imaging
Computer Science
Detectors
Face recognition
Image detection
Image Processing and Computer Vision
Landmarks
Modelling
Neural networks
Pattern Recognition
Pattern Recognition and Graphics
Test procedures
Three dimensional models
Vision
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We present a novel method for modeling 3D face shape, viewpoint, and expression from a single, unconstrained photo. Our method uses three deep convolutional neural networks to estimate each of these components separately. Importantly, unlike others, our method does not use facial landmark detection at test time; instead, it estimates these properties directly from image intensities. In fact, rather than using detectors, we show how accurate landmarks can be obtained as a by-product of our modeling process. We rigorously test our proposed method. To this end, we raise a number of concerns with existing practices used in evaluating face landmark detection methods. In response to these concerns, we propose novel paradigms for testing the effectiveness of rigid and non-rigid face alignment methods without relying on landmark detection benchmarks. We evaluate rigid face alignment by measuring its effects on face recognition accuracy on the challenging IJB-A and IJB-B benchmarks. Non-rigid, expression estimation is tested on the CK+ and EmotiW’17 benchmarks for emotion classification. We do, however, report the accuracy of our approach as a landmark detector for 3D landmarks on AFLW2000-3D and 2D landmarks on 300W and AFLW-PIFA. A surprising conclusion of these results is that better landmark detection accuracy does not necessarily translate to better face processing. Parts of this paper were previously published by Tran et al. ( 2017 ) and Chang et al. ( 2017 , 2018 ).
ISSN:0920-5691
1573-1405
DOI:10.1007/s11263-019-01151-x