Deep 3D caricature face generation with identity and structure consistency

This paper proposed a novel approach to generate face caricatures automatically from a single portrait image. We decompose the process of 3D face caricatures generation into two independent subtasks: appearance transfer of texture and the geometry transfer of mesh. For the appearance transfer, we de...

Full description

Saved in:
Bibliographic Details
Published in:Neurocomputing (Amsterdam) 2021-09, Vol.454, p.178-188
Main Authors: Li, Song, Su, Songzhi, Lin, Juncong, Cai, Guorong, Sun, Li
Format: Article
Language:English
Subjects:
3D face mesh
Caricature
Generative Adversarial Networks
Style transfer
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper proposed a novel approach to generate face caricatures automatically from a single portrait image. We decompose the process of 3D face caricatures generation into two independent subtasks: appearance transfer of texture and the geometry transfer of mesh. For the appearance transfer, we design a GAN-based network named CariFaceGAN to learn the style mapping from portrait to caricature, in which facial features are leveraged to preserve identity consistency. For geometry transfer, we first learn the transformation of the landmarks between portraits and caricatures in an embedded space obtained with Locally Linear Embedding method, and then Kriging interpolation is used to manipulate the portrait mesh constructed from a single image. The experimental results show that our proposed CariFaceGAN outperforms the state-of-the-art methods in terms of maintaining identity consistency and providing satisfactory visual effects.
ISSN:0925-2312
1872-8286
DOI:10.1016/j.neucom.2021.05.014