Hybrid Architectures Ensemble Learning for pseudo-label refinement in semi-supervised segmentation

• Published in: Information fusion 2025-04, Vol.116, p.102791, Article 102791
• Main Authors: Yang, Rui, Bai, Yunfei, Liu, Chang, Liu, Yuehua, Li, Xiaomao, Xie, Shaorong
• Format: Article
• Language: English
• Subjects: Channel dropout; Consistency regularization; Hybrid Architectures; Mean teacher; Semi-Supervised Semantic Segmentation
• ISSN: 1566-2535
• DOI: 10.1016/j.inffus.2024.102791

The performance of semi-supervised semantic segmentation models is significantly influenced by pseudo-label. To enhance the quality of pseudo-labels, we propose the Hybrid Architectures Ensemble Learning (HAEL) method. Specifically, we observe that different network architectures excel in specific tasks. CNN-based models are adept at capturing fine-grained features, whereas ViT-based models excel in perceiving significant intra-class variations and long-range dependencies. To integrate the diversity and specificity of these architectures, we propose a Hybrid Teacher–Student Model (HTM) incorporating two teacher–student branches. Each branch dynamically adjusts its prediction weights based on its expertise, thereby refining pseudo-labels by enhancing fine-grained feature representations and long-range dependencies. Furthermore, to enhance pseudo-labels’ stability and encourage them to learn model-specific features, we introduce a Similarity-Guided Channel Dropout (SCD) mechanism. This mechanism retains features shared by teachers with higher probability, mitigating the degradation of pseudo-label quality associated with the loss of important feature channels discarded by classical channel dropout. Meanwhile, unlike HTM, which explicitly combines the masks of the two architectures based on confidence maps, SCD implicitly explores model-specific features through rigorous screening based on each teacher’s unique knowledge. Finally, a Blind Spot Constraint (BSC) is introduced to impose additional penalties to improve the accuracy of hard samples in pseudo-labels. Extensive experiments on the Pascal VOC 2012 and Cityscapes demonstrate the effectiveness of the HAEL, achieving significant performance improvements. •Improves pseudo–label accuracy by utilizing multi–architectures to explore diverse features.•Stabilizes pseudo–labels and encourages learning of model–specific features.•Boosts pseudo–label confidence by penalizing all–teacher misclassified regions.•Achieves SOTA results on the Cityscapes and Pascal VOC 2012 datasets.