ADPL: Adaptive Dual Path Learning for Domain Adaptation of Semantic Segmentation

• Published in: IEEE transactions on pattern analysis and machine intelligence 2023-08, Vol.45 (8), p.9339-9356
• Main Authors: Cheng, Yiting, Wei, Fangyun, Bao, Jianmin, Chen, Dong, Zhang, Wenqiang
• Format: Article
• Language: English
• Subjects: Adaptation; Adaptation models; Data models; Domain adaptation; Image annotation; Image segmentation; image translation; Labels; Pipelines; Self-supervised learning; Semantic segmentation; Semantics; Synthetic data; Training; Visualization
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2023.3248294

To alleviate the need for large-scale pixel-wise annotations, domain adaptation for semantic segmentation trains segmentation models on synthetic data (source) with computer-generated annotations, which can be then generalized to segment realistic images (target). Recently, self-supervised learning (SSL) with a combination of image-to-image translation shows great effectiveness in adaptive segmentation. The most common practice is to perform SSL along with image translation to well align a single domain (source or target). However, in this single-domain paradigm, unavoidable visual inconsistency raised by image translation may affect subsequent learning. In addition, pseudo labels generated by a single segmentation model aligned in either the source or target domain may be not accurate enough for SSL. In this paper, based on the observation that domain adaptation frameworks performed in the source and target domain are almost complementary, we propose a novel adaptive dual path learning (ADPL) framework to alleviate visual inconsistency and promote pseudo-labeling by introducing two interactive single-domain adaptation paths aligned in source and target domain respectively. To fully explore the potential of this dual-path design, novel technologies such as dual path image translation (DPIT), dual path adaptive segmentation (DPAS), dual path pseudo label generation (DPPLG) and Adaptive ClassMix are proposed. The inference of ADPL is extremely simple, only one segmentation model in the target domain is employed. Our ADPL outperforms the state-of-the-art methods by large margins on GTA5\rightarrow → Cityscapes, SYNTHIA \rightarrow → Cityscapes and GTA5\rightarrow → BDD100K scenarios. Code and models are available at https://github.com/royee182/DPL .