UM-CAM: Uncertainty-weighted multi-resolution class activation maps for weakly-supervised segmentation

• Published in: Pattern recognition 2025-04, Vol.160, p.111204, Article 111204
• Main Authors: Fu, Jia, Wang, Guotai, Lu, Tao, Yue, Qiang, Vercauteren, Tom, Ourselin, Sébastien, Zhang, Shaoting
• Format: Article
• Language: English
• Subjects: Brain tumor; Class activation map; Exponential geodesic distance; Noisy label; Segmentation
• ISSN: 0031-3203
• DOI: 10.1016/j.patcog.2024.111204

Weakly-supervised medical image segmentation methods utilizing image-level labels have gained attention for reducing the annotation cost. They typically use Class Activation Maps (CAM) from a classification network but struggle with incomplete activation regions due to low-resolution localization without detailed boundaries. Differently from most of them that only focus on improving the quality of CAMs, we propose a more unified weakly-supervised segmentation framework with image-level supervision. Firstly, an Uncertainty-weighted Multi-resolution Class Activation Map (UM-CAM) is proposed to generate high-quality pixel-level pseudo-labels. Subsequently, a Geodesic distance-based Seed Expansion (GSE) strategy is introduced to rectify ambiguous boundaries in the UM-CAM by leveraging contextual information. To train a final segmentation model from noisy pseudo-labels, we introduce a Random-View Consensus (RVC) training strategy to suppress unreliable pixel/voxels and encourage consistency between random-view predictions. Extensive experiments on 2D fetal brain segmentation and 3D brain tumor segmentation tasks showed that our method significantly outperforms existing weakly-supervised methods. Code is available at: https://github.com/HiLab-git/UM-CAM. •A novel weakly-supervised segmentation method learning from image-level labels.•Uncertainty-weighted multi-resolution class activation map to generate pixel-level pseudo-labels.•Geodesic distance-based seed expansion to generate more accurate pseudo labels.•Random-view consensus to learn from noisy pseudo-labels.•The method shows great potential in achieving accurate segmentation with low annotation costs.