Three-dimensional affinity learning based multi-branch ensemble network for breast tumor segmentation in MRI

• Published in: Pattern recognition 2022-09, Vol.129, p.108723, Article 108723
• Main Authors: Zhou, Lei, Wang, Shuai, Sun, Kun, Zhou, Tao, Yan, Fuhua, Shen, Dinggang
• Format: Article
• Language: English
• Subjects: Breast tumor segmentation; Multi-branch ensemble network; Three-dimensional affinity learning based refinement
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2022.108723

•We propose a multi-branch network for breast tumor segmentation.•A trainable 3D affinity learning based segmentation refinement module is designed.•A hierarchial ensemble strategy is designed to combine two subnetworks comprehensively.•The state-of-the-art performance for breast tumor segmentation in MRI is achieved. Accurate and automatic breast tumor segmentation based on dynamic contrast-enhancement magnetic resonance imaging (DCE-MRI) plays an important role in breast cancer analysis. However, the background parenchymal enhancement and large variations in tumor size, shape or appearance make the task very challenging, and also the segmentation performance is still not satisfactory, especially for non-mass enhancement (NME) and small size tumors (≤2 cm). To address these challenges, we propose a novel 3D affinity learning based multi-branch ensemble network for accurate breast tumor segmentation. Specifically, two different types of subnetworks are built to form a multi-branch network. The two subnetworks are equipped with effective operation components, i.e., residual connection and channel-wise attention or making use of dense connectivity patterns, which can process the input images in parallel. Second, we propose an end-to-end trainable 3D affinity learning based refinement module by calculating the similarities between features of voxels, which is useful in discovering more pixels belonging to breast tumors. Third, two local affinity matrices are constructed by 3D affinity learning, which are used to refine the segmentation outputs of two subnetworks, respectively. Finally, a novel ensemble module is proposed to combine the information derived from the subnetworks, which can hierarchically merge the local and global affinity matrices derived from subnetworks. A large-scale breast DCE-MR images dataset with 420 subjects are built for evaluation, and comprehensive experiments have been conducted to demonstrate that our proposed method achieves superior performance over state-of-the-art medical image segmentation methods.