Plausible 3D Face Wrinkle Generation Using Variational Autoencoders

• Published in: IEEE transactions on visualization and computer graphics 2022-09, Vol.28 (9), p.3113-3125
• Main Authors: Deng, Qixin, Ma, Luming, Jin, Aobo, Bi, Huikun, Le, Binh Huy, Deng, Zhigang
• Format: Article
• Language: English
• Subjects: Animation; Computational modeling; Data models; deep generative models; Face modeling; Faces; Interpolation; Pipelines; Shape; Solid modeling; Synthesis; Three dimensional models; Three-dimensional displays; Topology; variational autoencoders; Virtual reality; wrinkle synthesis
• ISSN: 1077-2626; 1941-0506
• DOI: 10.1109/TVCG.2021.3051251

Realistic 3D facial modeling and animation have been increasingly used in many graphics, animation, and virtual reality applications. However, generating realistic fine-scale wrinkles on 3D faces, in particular, on animated 3D faces, is still a challenging problem that is far away from being resolved. In this article we propose an end-to-end system to automatically augment coarse-scale 3D faces with synthesized fine-scale geometric wrinkles. By formulating the wrinkle generation problem as a supervised generation task, we implicitly model the continuous space of face wrinkles via a compact generative model, such that plausible face wrinkles can be generated through effective sampling and interpolation in the space. We also introduce a complete pipeline to transfer the synthesized wrinkles between faces with different shapes and topologies. Through many experiments, we demonstrate our method can robustly synthesize plausible fine-scale wrinkles on a variety of coarse-scale 3D faces with different shapes and expressions.