Learning to Search a Lightweight Generalized Network for Medical Image Fusion

Image fusion is indispensable in a comprehensive medical imaging pipeline. By embracing deep learning technology, medical image fusion has achieved tremendous progress over the past few years. However, existing approaches make efforts on the specific type of medical image fusion task and may face di...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on circuits and systems for video technology 2024-07, Vol.34 (7), p.5921-5934
Main Authors: Mu, Pan, Wu, Guanyao, Liu, Jinyuan, Zhang, Yuduo, Fan, Xin, Liu, Risheng
Format: Article
Language:English
Subjects:
Complexity
Computer architecture
Computer vision
Deep learning
Feature extraction
Hardware
Image fusion
Image segmentation
Magnetic resonance imaging
Medical diagnostic imaging
Medical image fusion
medical image processing
Medical imaging
Microprocessors
Modules
neural architecture search
Searching
Source code
Task analysis
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Image fusion is indispensable in a comprehensive medical imaging pipeline. By embracing deep learning technology, medical image fusion has achieved tremendous progress over the past few years. However, existing approaches make efforts on the specific type of medical image fusion task and may face difficulties in generalizing well. Moreover, most of them strain every nerve to design various architectures with an increase of the width of depth, placing an obstacle in running efficiency. To address the above problems, we propose an Auto-searching Light-weighted Multi-source Fusion network, namely ALMFnet, aiming at incorporating both software and hardware knowledge in a network architecture searching manner for medical image fusion. Specifically, the ALMFnet, consisting of two different feature-extracting modules and one fusion module, is developed to extract and refine multi-source features in a generalized model. Besides, motivated by the collaborative principle, we introduce hardware constraints for sufficient searching the each particular component, further reducing the complexity of the obtained model. Furthermore, to preserve important details in pathological image areas, we introduce a segmentation mask into the developed method. Experimental results demonstrate that our generalized model outperforms previous methods not only in terms of quantitative scores but also in model complexity. Source code will be available at https://github.com/RollingPlain/ALMFnet .
ISSN:1051-8215
1558-2205
DOI:10.1109/TCSVT.2023.3342808