Face Alignment in Full Pose Range: A 3D Total Solution

• Published in: IEEE transactions on pattern analysis and machine intelligence 2019-01, Vol.41 (1), p.78-92
• Main Authors: Zhu, Xiangyu, Liu, Xiaoming, Lei, Zhen, Li, Stan Z.
• Format: Article
• Language: English
• Subjects: 3D morphable model; Alignment; Artificial neural networks; cascaded regression; Computer vision; convolutional neural network; Face; Face alignment; Landmarks; Shape; Solid modeling; State of the art; Three dimensional models; Three-dimensional displays; Training; Two dimensional displays; Yaw
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2017.2778152

Face alignment, which fits a face model to an image and extracts the semantic meanings of facial pixels, has been an important topic in the computer vision community. However, most algorithms are designed for faces in small to medium poses (yaw angle is smaller than 45 degree), which lack the ability to align faces in large poses up to 90 degree. The challenges are three-fold. First, the commonly used landmark face model assumes that all the landmarks are visible and is therefore not suitable for large poses. Second, the face appearance varies more drastically across large poses, from the frontal view to the profile view. Third, labelling landmarks in large poses is extremely challenging since the invisible landmarks have to be guessed. In this paper, we propose to tackle these three challenges in an new alignment framework termed 3D Dense Face Alignment (3DDFA), in which a dense 3D Morphable Model (3DMM) is fitted to the image via Cascaded Convolutional Neural Networks. We also utilize 3D information to synthesize face images in profile views to provide abundant samples for training. Experiments on the challenging AFLW database show that the proposed approach achieves significant improvements over the state-of-the-art methods.