D-TrAttUnet: Toward hybrid CNN-transformer architecture for generic and subtle segmentation in medical images

• Published in: Computers in biology and medicine 2024-06, Vol.176, p.108590, Article 108590
• Main Authors: Bougourzi, Fares, Dornaika, Fadi, Distante, Cosimo, Taleb-Ahmed, Abdelmalik
• Format: Article
• Language: English
• Subjects: Adaptability; Bone Metastasis; Coders; Computer Science; Convolutional Neural Network; COVID-19; COVID-19 - diagnostic imaging; Decoders; Deep learning; Humans; Image analysis; Image Interpretation, Computer-Assisted - methods; Image processing; Image Processing, Computer-Assisted - methods; Image segmentation; Lesions; Machine Learning; Medical Imaging; Metastases; Neural Networks, Computer; Nuclei; Organs; SARS-CoV-2; Segmentation; Transformer; Unet
• ISSN: 0010-4825; 1879-0534; 1879-0534
• DOI: 10.1016/j.compbiomed.2024.108590

Over the past two decades, machine analysis of medical imaging has advanced rapidly, opening up significant potential for several important medical applications. As complicated diseases increase and the number of cases rises, the role of machine-based imaging analysis has become indispensable. It serves as both a tool and an assistant to medical experts, providing valuable insights and guidance. A particularly challenging task in this area is lesion segmentation, a task that is challenging even for experienced radiologists. The complexity of this task highlights the urgent need for robust machine learning approaches to support medical staff. In response, we present our novel solution: the D-TrAttUnet architecture. This framework is based on the observation that different diseases often target specific organs. Our architecture includes an encoder–decoder structure with a composite Transformer-CNN encoder and dual decoders. The encoder includes two paths: the Transformer path and the Encoders Fusion Module path. The Dual-Decoder configuration uses two identical decoders, each with attention gates. This allows the model to simultaneously segment lesions and organs and integrate their segmentation losses. To validate our approach, we performed evaluations on the Covid-19 and Bone Metastasis segmentation tasks. We also investigated the adaptability of the model by testing it without the second decoder in the segmentation of glands and nuclei. The results confirmed the superiority of our approach, especially in Covid-19 infections and the segmentation of bone metastases. In addition, the hybrid encoder showed exceptional performance in the segmentation of glands and nuclei, solidifying its role in modern medical image analysis. •We propose an Efficient Hybrid CNN-Transformer architecture.•Our proposed approach Encoder has Transformer path and Encoders Fusion.•Our approach is designed with Multi-tasks infection-organ-attention Decoders.•The performance is evaluated for Segmenting Bone Metastases and Covid-19 infection.•The proposed Hybrid Encoder has been evaluated for Segmenting Glands and Nuclei.