Multi-Density Sketch-to-Image Translation Network

Sketch-to-image (S2I) translation plays an important role in image synthesis and manipulation tasks, such as photo editing and colorization. Some specific S2I translations, including sketch-to-photo and sketch-to-painting, can be used as powerful tools in the art design industry. However, previous m...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on multimedia 2022, Vol.24, p.4002-4015
Main Authors: Huang, Jialu, Jing, Liao, Tan, Zhifeng, Kwong, Sam
Format: Article
Language:English
Subjects:
Anime
Codes
Colorization
Decoding
Deep image synthesis
Density
Editing
Faces
GAN
Image edge detection
Image manipulation
Image synthesis
interactive editing
multi-scale disentangle
Sketches
Task analysis
Training
Translations
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Sketch-to-image (S2I) translation plays an important role in image synthesis and manipulation tasks, such as photo editing and colorization. Some specific S2I translations, including sketch-to-photo and sketch-to-painting, can be used as powerful tools in the art design industry. However, previous methods only support S2I translation with a single level of density, which gives less flexibility to users for controlling the input sketches. In this work, we propose the first multi-level density sketch-to-image translation framework, which allows the input sketch to cover a wide range from rough object outlines to microstructures. Moreover, to tackle the problem of noncontinuous representation of multi-level density input sketches, we project the density level into a continuous latent space, which can then be linearly controlled by a parameter. This allows users to conveniently control the densities of input sketches and the generation of images. Moreover, our method has been successfully verified on various datasets for different applications, including face editing, multi-modal sketch-to-photo translation, and anime colorization, providing coarse-to-fine levels of controls to these applications.
ISSN:1520-9210
1941-0077
DOI:10.1109/TMM.2021.3111501