ScribSD+: Scribble-supervised medical image segmentation based on simultaneous multi-scale knowledge distillation and class-wise contrastive regularization

• Published in: Computerized medical imaging and graphics 2024-09, Vol.116, p.102416, Article 102416
• Main Authors: Qu, Yijie, Lu, Tao, Zhang, Shaoting, Wang, Guotai
• Format: Article
• Language: English
• Subjects: Contrastive learning; Fetal MRI; Knowledge distillation; Weakly supervised learning
• ISSN: 0895-6111; 1879-0771; 1879-0771
• DOI: 10.1016/j.compmedimag.2024.102416

Despite that deep learning has achieved state-of-the-art performance for automatic medical image segmentation, it often requires a large amount of pixel-level manual annotations for training. Obtaining these high-quality annotations is time-consuming and requires specialized knowledge, which hinders the widespread application that relies on such annotations to train a model with good segmentation performance. Using scribble annotations can substantially reduce the annotation cost, but often leads to poor segmentation performance due to insufficient supervision. In this work, we propose a novel framework named as ScribSD+ that is based on multi-scale knowledge distillation and class-wise contrastive regularization for learning from scribble annotations. For a student network supervised by scribbles and the teacher based on Exponential Moving Average (EMA), we first introduce multi-scale prediction-level Knowledge Distillation (KD) that leverages soft predictions of the teacher network to supervise the student at multiple scales, and then propose class-wise contrastive regularization which encourages feature similarity within the same class and dissimilarity across different classes, thereby effectively improving the segmentation performance of the student network. Experimental results on the ACDC dataset for heart structure segmentation and a fetal MRI dataset for placenta and fetal brain segmentation demonstrate that our method significantly improves the student’s performance and outperforms five state-of-the-art scribble-supervised learning methods. Consequently, the method has a potential for reducing the annotation cost in developing deep learning models for clinical diagnosis. •An efficient scribble-supervised medical image segmentation framework by leveraging multi-scale knowledge distillation and class-wise contrastive regularization.•A Multi-scale Prediction Distillation (Ms-PD) strategy to learn from pseudo labels by using the noise-robust soft predictions obtained by the teacher at multiple scales to supervise the student.•A Multi-scale class-wise Contrastive Regularization (Ms-CR) strategy to learn distinctive feature representations for different classes by encouraging intra-class feature similarity and inter-class feature dissimilarity.•The proposed method outperformed five state-of-the-art weakly-supervised methods learning from scribble annotations on two medical image datasets, and it shows great potential in achieving accurate medical image