Toward few-shot domain adaptation with perturbation-invariant representation and transferable prototypes

• Published in: Frontiers of Computer Science 2022-06, Vol.16 (3), p.163347, Article 163347
• Main Authors: FAN, Junsong, WANG, Yuxi, GUAN, He, SONG, Chunfeng, ZHANG, Zhaoxiang
• Format: Article
• Language: English
• Subjects: Adaptation; Annotations; Computer Science; Deep learning; domain adaptation; Experiments; Invariants; Knowledge management; Labels; Perturbation; Prototypes; Representations; Research Article; Semantic segmentation; Semantics; Supervision
• ISSN: 2095-2228; 2095-2236
• DOI: 10.1007/s11704-022-2015-7

Domain adaptation (DA) for semantic segmentation aims to reduce the annotation burden for the dense pixel-level prediction task. It focuses on tackling the domain gap problem and manages to transfer knowledge learned from abundant source data to new target scenes. Although recent works have achieved rapid progress in this field, they still underperform fully supervised models with a large margin due to the absence of any available hints in the target domain. Considering that few-shot labels are cheap to obtain in practical applications, we attempt to leverage them to mitigate the performance gap between DA and fully supervised methods. The key to this problem is to leverage the few-shot labels to learn robust domain-invariant predictions effectively. To this end, we first design a data perturbation strategy to enhance the robustness of the representations. Furthermore, a transferable prototype module is proposed to bridge the domain gap based on the source data and few-shot targets. By means of these proposed methods, our approach can perform on par with the fully supervised models to some extent. We conduct extensive experiments to demonstrate the effectiveness of the proposed methods and report the state-of-the-art performance on two popular DA tasks, i.e., from GTA5 to Cityscapes and SYNTHIA to Cityscapes.