ACN: Occlusion-tolerant face alignment by attentional combination of heterogeneous regression networks

• Published in: Pattern recognition 2021-06, Vol.114, p.107761, Article 107761
• Main Authors: Park, Hyunsung, Kim, Daijin
• Format: Article
• Language: English
• Subjects: Attentional combination network; Converting networks; Face alignment; Facial landmark localization
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2020.107761

•We propose an occlusion-tolerant highly accurate face alignment method.•It combines a coordinate and a heatmap regression network with a spatial attention.•It compensates complementarily overall fitting tendency and detailed localization.•It uses coordinate-to-heatmap and heatmap-to-coordinate conversion networks.•It achieves state-of-the-art accuracy In experiments on several benchmarks. This paper presents the Attentional Combination Network (ACN), which is a highly accurate face alignment method that is tolerant of occlusion. The method combines a coordinate regression network and a heatmap regression network with a spatial attention. The coordinate regression generates the coordinates of facial landmark points directly such that they are fitted to the input face on the whole. The heatmap regression generates the heatmap of facial landmark points such that each channel provides good localization of the detail of its facial landmark point. These independent regressions compensate for each other complementarily such that the overall fitting tendency of the coordinate regression compensates for the inaccurate alignment of the heatmap regression due to missing local information, and the detailed localization of the heatmap regression compensates for the relatively inaccurate alignment of the coordinate regression. The proposed ACN uses coordinate-to-heatmap and the heatmap-to-coordinate conversion networks to combine two heterogeneous regressions, and to generate the final coordinates of the facial landmark points. The ACN use the spatial attention mechanism to effectively reject impeditive local features that are caused by the occlusion. In experiments on several benchmarks, the proposed ACN achieved state-of-the-art accuracy