SwinPA-Net: Swin Transformer-Based Multiscale Feature Pyramid Aggregation Network for Medical Image Segmentation

• Published in: IEEE transaction on neural networks and learning systems 2024-04, Vol.35 (4), p.5355-5366
• Main Authors: Du, Hao, Wang, Jiazheng, Liu, Min, Wang, Yaonan, Meijering, Erik
• Format: Article
• Language: English
• Subjects: Aggregation; Ambient noise; Background noise; Benchmarking; Datasets; Dense multiplicative connection (DMC) module; Image processing; Image Processing, Computer-Assisted; Image segmentation; Laparoscopy; Learning; Lesions; local pyramid attention (LPA) module; Medical diagnostic imaging; medical image segmentation; Medical imaging; Modules; Monte Carlo methods; Neural Networks, Computer; Pathology; Polyps; Semantics; Skin diseases; Skin lesions; Task analysis; Transformers
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2022.3204090

The precise segmentation of medical images is one of the key challenges in pathology research and clinical practice. However, many medical image segmentation tasks have problems such as large differences between different types of lesions and similar shapes as well as colors between lesions and surrounding tissues, which seriously affects the improvement of segmentation accuracy. In this article, a novel method called Swin Pyramid Aggregation network (SwinPA-Net) is proposed by combining two designed modules with Swin Transformer to learn more powerful and robust features. The two modules, named dense multiplicative connection (DMC) module and local pyramid attention (LPA) module, are proposed to aggregate the multiscale context information of medical images. The DMC module cascades the multiscale semantic feature information through dense multiplicative feature fusion, which minimizes the interference of shallow background noise to improve the feature expression and solves the problem of excessive variation in lesion size and type. Moreover, the LPA module guides the network to focus on the region of interest by merging the global attention and the local attention, which helps to solve similar problems. The proposed network is evaluated on two public benchmark datasets for polyp segmentation task and skin lesion segmentation task as well as a clinical private dataset for laparoscopic image segmentation task. Compared with existing state-of-the-art (SOTA) methods, the SwinPA-Net achieves the most advanced performance and can outperform the second-best method on the mean Dice score by 1.68%, 0.8%, and 1.2% on the three tasks, respectively.