HQ3DAvatar: High-quality Implicit 3D Head Avatar

• Published in: ACM transactions on graphics 2024-06, Vol.43 (3), p.1-24, Article 27
• Main Authors: Teotia, Kartik, R, Mallikarjun B, Pan, Xingang, Kim, Hyeongwoo, Garrido, Pablo, Elgharib, Mohamed, Theobalt, Christian
• Format: Article
• Language: English
• Subjects: Computing methodologies; Rendering; Volumetric models
• ISSN: 0730-0301; 1557-7368
• DOI: 10.1145/3649889

Multi-view volumetric rendering techniques have recently shown great potential in modeling and synthesizing high-quality head avatars. A common approach to capture full head dynamic performances is to track the underlying geometry using a mesh-based template or 3D cube-based graphics primitives. While these model-based approaches achieve promising results, they often fail to learn complex geometric details such as the mouth interior, hair, and topological changes over time. This article presents a novel approach to building highly photorealistic digital head avatars. Our method learns a canonical space via an implicit function parameterized by a neural network. It leverages multiresolution hash encoding in the learned feature space, allowing for high quality, faster training, and high-resolution rendering. At test time, our method is driven by a monocular RGB video. Here, an image encoder extracts face-specific features that also condition the learnable canonical space. This encourages deformation-dependent texture variations during training. We also propose a novel optical flow-based loss that ensures correspondences in the learned canonical space, thus encouraging artifact-free and temporally consistent renderings. We show results on challenging facial expressions and show free-viewpoint renderings at interactive real-time rates for a resolution of 480x270. Our method outperforms related approaches both visually and numerically. We will release our multiple-identity dataset to encourage further research.