Unsupervised domain adaptive segmentation algorithm based on two-level category alignment

• Published in: Neural networks 2024-09, Vol.177, p.106399, Article 106399
• Main Authors: Dong, Wenyong, Liang, Zhixue, Wang, Liping, Tian, Gang, Long, Qianhui
• Format: Article
• Language: English
• Subjects: Class activation map; Generative adversarial network; Semantic segmentation; Two-level category alignment; Unsupervised domain adaption
• ISSN: 0893-6080; 1879-2782; 1879-2782
• DOI: 10.1016/j.neunet.2024.106399

To enhance the model’s generalization ability in unsupervised domain adaptive segmentation tasks, most approaches have primarily focused on pixel-level local features, but neglected the clue in category information. This limitation results in the segmentation network only learning global inter-domain invariant features but ignoring the category-specific inter-domain invariant features, which degenerates the segmentation performance. To address this issue, we present an Unsupervised Domain Adaptive algorithm based on two-level Category Alignment in two different spaces for semantic segmentation tasks, denoted as UDAca+. The first level is image-level category alignment based on class activation map (CAM), and the second one is pixel-level category alignment based on pseudo label. By utilizing category information, UDAca+ can effectively capture domain-invariant yet category-discriminative feature representations to improve segmentation accuracy. In addition, an adversarial learning-based strategy in mixed domain is designed to train the proposed network. Moreover, a confidence calculation method is introduced to mitigate the misleading issues of negative transfer and over-alignment caused by the noise in image-level pseudo labels. UDAca+ achieves the state-of-the-art (SOTA) performance on two synthetic-to-real adaptative tasks, and verifies its effectiveness for image segmentation. •A UDA algorithm is proposed to leverage the clue provided by category information.•The strategy aids in learning category-discriminative feature representations.•The mixed domain can further improve the effectiveness of adversarial learning.•Extensive experiments demonstrate the superiority of our proposed method.