IDE-3D: Interactive Disentangled Editing for High-Resolution 3D-Aware Portrait Synthesis

Existing 3D-aware facial generation methods face a dilemma in quality versus editability: they either generate editable results in low resolution, or high-quality ones with no editing flexibility. In this work, we propose a new approach that brings the best of both worlds together. Our system consis...

Full description

Saved in:
Bibliographic Details
Published in:ACM transactions on graphics 2022-12, Vol.41 (6), p.1-10, Article 270
Main Authors: Sun, Jingxiang, Wang, Xuan, Shi, Yichun, Wang, Lizhen, Wang, Jue, Liu, Yebin
Format: Article
Language:English
Subjects:
Computer graphics
Computing methodologies
Image manipulation
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Existing 3D-aware facial generation methods face a dilemma in quality versus editability: they either generate editable results in low resolution, or high-quality ones with no editing flexibility. In this work, we propose a new approach that brings the best of both worlds together. Our system consists of three major components: (1) a 3D-semantics-aware generative model that produces view-consistent, disentangled face images and semantic masks; (2) a hybrid GAN inversion approach that initializes the latent codes from the semantic and texture encoder, and further optimizes them for faithful reconstruction; and (3) a canonical editor that enables efficient manipulation of semantic masks in canonical view and produces high-quality editing results. Our approach is competent for many applications, e.g. free-view face drawing, editing and style control. Both quantitative and qualitative results show that our method reaches the state-of-the-art in terms of photorealism, faithfulness and efficiency.
ISSN:0730-0301
1557-7368
DOI:10.1145/3550454.3555506