Multi-modal contrastive mutual learning and pseudo-label re-learning for semi-supervised medical image segmentation

• Published in: Medical image analysis 2023-01, Vol.83, p.102656-102656, Article 102656
• Main Authors: Zhang, Shuo, Zhang, Jiaojiao, Tian, Biao, Lukasiewicz, Thomas, Xu, Zhenghua
• Format: Article
• Language: English
• Subjects: Humans; Medical image segmentation; Multi-modal contrastive learning; Pseudo-label re-learning; Semi-supervised learning; Supervised Machine Learning
• ISSN: 1361-8415; 1361-8423
• DOI: 10.1016/j.media.2022.102656

Semi-supervised learning has a great potential in medical image segmentation tasks with a few labeled data, but most of them only consider single-modal data. The excellent characteristics of multi-modal data can improve the performance of semi-supervised segmentation for each image modality. However, a shortcoming for most existing multi-modal solutions is that as the corresponding processing models of the multi-modal data are highly coupled, multi-modal data are required not only in the training but also in the inference stages, which thus limits its usage in clinical practice. Consequently, we propose a semi-supervised contrastive mutual learning (Semi-CML) segmentation framework, where a novel area-similarity contrastive (ASC) loss leverages the cross-modal information and prediction consistency between different modalities to conduct contrastive mutual learning. Although Semi-CML can improve the segmentation performance of both modalities simultaneously, there is a performance gap between two modalities, i.e., there exists a modality whose segmentation performance is usually better than that of the other. Therefore, we further develop a soft pseudo-label re-learning (PReL) scheme to remedy this gap. We conducted experiments on two public multi-modal datasets. The results show that Semi-CML with PReL greatly outperforms the state-of-the-art semi-supervised segmentation methods and achieves a similar (and sometimes even better) performance as fully supervised segmentation methods with 100% labeled data, while reducing the cost of data annotation by 90%. We also conducted ablation studies to evaluate the effectiveness of the ASC loss and the PReL module. [Display omitted] •Propose a new low-coupling multi-modal semi-supervised segmentation framework.•The inference of the proposed Semi-CML with PReL only needs one modality data.•Propose a new ASC loss to utilize area context information in contrastive learning.•A new soft pseudo-label re-learning to narrow performance gaps between modalities.•This work greatly outperforms the SOTA semi-supervised segmentation baselines.