SwapInpaint2: Towards high structural consistency in identity-guided inpainting via background-preserving GAN inversion

In this work, we propose SwapInpaint2 to enhance the naturalness of identity-guided face inpainting. The previous version, SwapInpaint, relied on a generic inpainting model for content infer, bringing the problems of unnatural structure to the results. Our method addresses this issue by utilizing an...

Full description

Saved in:
Bibliographic Details
Published in:Pattern recognition 2025-02, Vol.158, p.110969, Article 110969
Main Authors: Li, Honglei, Zhang, Yifan, Wang, Wenmin, Zhang, Shenyong, Zhang, Shixiong
Format: Article
Language:English
Subjects:
GAN inversion
Identity-guided face inpainting
Image synthesis
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this work, we propose SwapInpaint2 to enhance the naturalness of identity-guided face inpainting. The previous version, SwapInpaint, relied on a generic inpainting model for content infer, bringing the problems of unnatural structure to the results. Our method addresses this issue by utilizing an inversion-based Background-preserving Attribute Extractor and an improved Embedding Integration Generator to provide high-quality attribute embeddings and produce high-naturalistic results. Comparison experiments with state-of-the-art models demonstrate that our new method achieves superior structural consistency and stronger style alignment both in terms of quality and quantity. •Structure-consistent attribute embeddings can improve the naturalness of inpainted results.•A inversion-based background-preserving attribute extractor.•Injecting background to modulation blocks improves style blending.
ISSN:0031-3203
DOI:10.1016/j.patcog.2024.110969