UNETR++: Delving Into Efficient and Accurate 3D Medical Image Segmentation

• Published in: IEEE transactions on medical imaging 2024-09, Vol.43 (9), p.3377-3390
• Main Authors: Shaker, Abdelrahman, Maaz, Muhammad, Rasheed, Hanoona, Khan, Salman, Yang, Ming-Hsuan, Shahbaz Khan, Fahad
• Format: Article
• Language: English
• Subjects: Algorithms; Biomedical imaging; Complexity theory; Databases, Factual; Deep Learning; efficient attention; Graphics processing units; Humans; hybrid architecture; Image segmentation; Imaging, Three-Dimensional - methods; medical image segmentation; Task analysis; Three-dimensional displays; Tomography, X-Ray Computed - methods; Transformers
• ISSN: 0278-0062; 1558-254X; 1558-254X
• DOI: 10.1109/TMI.2024.3398728

Owing to the success of transformer models, recent works study their applicability in 3D medical segmentation tasks. Within the transformer models, the self-attention mechanism is one of the main building blocks that strives to capture long-range dependencies, compared to the local convolutional-based design. However, the self-attention operation has quadratic complexity which proves to be a computational bottleneck, especially in volumetric medical imaging, where the inputs are 3D with numerous slices. In this paper, we propose a 3D medical image segmentation approach, named UNETR++, that offers both high-quality segmentation masks as well as efficiency in terms of parameters, compute cost, and inference speed. The core of our design is the introduction of a novel efficient paired attention (EPA) block that efficiently learns spatial and channel-wise discriminative features using a pair of inter-dependent branches based on spatial and channel attention. Our spatial attention formulation is efficient and has linear complexity with respect to the input. To enable communication between spatial and channel-focused branches, we share the weights of query and key mapping functions that provide a complimentary benefit (paired attention), while also reducing the complexity. Our extensive evaluations on five benchmarks, Synapse, BTCV, ACDC, BraTS, and Decathlon-Lung, reveal the effectiveness of our contributions in terms of both efficiency and accuracy. On Synapse, our UNETR++ sets a new state-of-the-art with a Dice Score of 87.2%, while significantly reducing parameters and FLOPs by over 71%, compared to the best method in the literature. Our code and models are available at: https://tinyurl.com/2p87x5xn .