Exploring Uncertainty-Based Self-Prompt for Test-Time Adaptation Semantic Segmentation in Remote Sensing Images

Test-time adaptation (TTA) has been proven to effectively improve the adaptability of deep learning semantic segmentation models facing continuous changeable scenes. However, most of the existing TTA algorithms lack an explicit exploration of domain gaps, especially those based on visual domain prom...

Full description

Saved in:
Bibliographic Details
Published in:Remote sensing (Basel, Switzerland) Switzerland), 2024-04, Vol.16 (7), p.1239
Main Authors: Wang, Ziquan, Zhang, Yongsheng, Zhang, Zhenchao, Jiang, Zhipeng, Yu, Ying, Li, Lei, Zhang, Lei
Format: Article
Language:English
Subjects:
Adaptability
Adaptation
Algorithms
Artificial intelligence
Benchmarks
Comparative analysis
Datasets
Deep learning
Entropy
Image processing
Image segmentation
Knowledge
Learning
Machine learning
Methods
Modules
Neural networks
Remote sensing
Semantic segmentation
Semantics
Teachers
Technology application
test-time domain adaptation
Testing time
Uncertainty
uncertainty-based self-prompt
Vehicles
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Test-time adaptation (TTA) has been proven to effectively improve the adaptability of deep learning semantic segmentation models facing continuous changeable scenes. However, most of the existing TTA algorithms lack an explicit exploration of domain gaps, especially those based on visual domain prompts. To address these issues, this paper proposes a self-prompt strategy based on uncertainty, guiding the model to continuously focus on regions with high uncertainty (i.e., regions with a larger domain gap). Specifically, we still use the Mean-Teacher architecture with the predicted entropy from the teacher network serving as the input to the prompt module. The prompt module processes uncertain maps and guides the student network to focus on regions with higher entropy, enabling continuous adaptation to new scenes. This is a self-prompting strategy that requires no prior knowledge and is tested on widely used benchmarks. In terms of the average performance, our method outperformed the baseline algorithm in TTA and continual TTA settings of Cityscapes-to-ACDC by 3.3% and 3.9%, respectively. Our method also outperformed the baseline algorithm by 4.1% and 3.1% on the more difficult Cityscapes-to-(Foggy and Rainy) Cityscapes setting, which also surpasses six other current TTA methods.
ISSN:2072-4292
2072-4292
DOI:10.3390/rs16071239