Pixel-wise conditioned generative adversarial networks for image synthesis and completion

• Published in: Neurocomputing (Amsterdam) 2020-11, Vol.416, p.218-230
• Main Authors: Ruffino, Cyprien, Hérault, Romain, Laloy, Eric, Gasso, Gilles
• Format: Article
• Language: English
• Subjects: Artificial Intelligence; Computer Science; Conditional GAN; Deep generative models; Generative adversarial networks
• ISSN: 0925-2312; 1872-8286
• DOI: 10.1016/j.neucom.2019.11.116

Generative Adversarial Networks (GANs) have proven successful for unsupervised image generation. Several works have extended GANs to image inpainting by conditioning the generation with parts of the image to be reconstructed. Despite their success, these methods have limitations in settings where only a small subset of the image pixels is known beforehand. In this paper we investigate the effectiveness of conditioning GANs when very few pixel values are provided. We propose a modelling framework which results in adding an explicit cost term to the GAN objective function to enforce pixel-wise conditioning. We investigate the influence of this regularization term on the quality of the generated images and the fulfillment of the given pixel constraints. Using the recent PacGAN technique, we ensure that we keep diversity in the generated samples. Conducted experiments on FashionMNIST show that the regularization term effectively controls the trade-off between quality of the generated images and the conditioning. Experimental evaluation on the CIFAR-10 and CelebA datasets evidences that our method achieves accurate results both visually and quantitatively in term of Fréchet Inception Distance, while still enforcing the pixel conditioning. We also evaluate our method on a texture image generation task using fully-convolutional networks. As a final contribution, we apply the method to a classical geological simulation application.