WaveFill: A Wavelet-based Generation Network for Image Inpainting

• Main Authors: Yu, Yingchen, Zhan, Fangneng, Lu, Shijian, Pan, Jianxiong, Ma, Feiying, Xie, Xuansong, Miao, Chunyan
• Format: Conference Proceeding
• Language: English
• Subjects: Computer vision; Discrete wavelet transforms; Frequency synthesizers; Fuses; Generative adversarial networks; Hybrid power systems; Image and video synthesis; Neural generative models; Wavelet domain
• ISSN: 2380-7504
• DOI: 10.1109/ICCV48922.2021.01385

Image inpainting aims to complete the missing or corrupted regions of images with realistic contents. The prevalent approaches adopt a hybrid objective of reconstruction and perceptual quality by using generative adversarial networks. However, the reconstruction loss and adversarial loss focus on synthesizing contents of different frequencies and simply applying them together often leads to inter-frequency conflicts and compromised inpainting. This paper presents WaveFill, a wavelet-based inpainting network that decomposes images into multiple frequency bands and fills the missing regions in each frequency band separately and explicitly. WaveFill decomposes images by using discrete wavelet transform (DWT) that preserves spatial information naturally. It applies L1 reconstruction loss to the decomposed low-frequency bands and adversarial loss to high-frequency bands, hence effectively mitigate inter-frequency conflicts while completing images in spatial domain. To address the inpainting inconsistency in different frequency bands and fuse features with distinct statistics, we design a novel normalization scheme that aligns and fuses the multi-frequency features effectively. Extensive experiments over multiple datasets show that WaveFill achieves superior image inpainting qualitatively and quantitatively.