Multi-Domain Image-to-Image Translation with Cross-Granularity Contrastive Learning

• Published in: ACM transactions on multimedia computing communications and applications 2024-05, Vol.20 (7), p.1-21, Article 228
• Main Authors: Fu, Huiyuan, Liu, Jin, Yu, Ting, Wang, Xin, Ma, Huadong
• Format: Article
• Language: English
• Subjects: Artificial intelligence; Computer vision; Computing methodologies
• ISSN: 1551-6857; 1551-6865
• DOI: 10.1145/3656048

The objective of multi-domain image-to-image translation is to learn the mapping from a source domain to a target domain in multiple image domains while preserving the content representation of the source domain. Despite the importance and recent efforts, most previous studies disregard the large style discrepancy between images and instances in various domains, or fail to capture instance details and boundaries properly, resulting in poor translation results for rich scenes. To address these problems, we present an effective architecture for multi-domain image-to-image translation that only requires one generator. Specifically, we provide detailed procedures for capturing the features of instances throughout the learning process, as well as learning the relationship between the style of the global image and that of a local instance in the image by enforcing the cross-granularity consistency. In order to capture local details within the content space, we employ a dual contrastive learning strategy that operates at both the instance and patch levels. Extensive studies on different multi-domain image-to-image translation datasets reveal that our proposed method outperforms state-of-the-art approaches.