IMPORTANT-Net: Integrated MRI multi-parametric increment fusion generator with attention network for synthesizing absent data

• Published in: Information fusion 2024-08, Vol.108, p.102381, Article 102381
• Main Authors: Zhang, Tianyu, Tan, Tao, Han, Luyi, Wang, Xin, Gao, Yuan, van Dijk, Jarek, Portaluri, Antonio, Gonzalez-Huete, Abel, D’Angelo, Anna, Lu, Chunyao, Teuwen, Jonas, Beets-Tan, Regina, Sun, Yue, Mann, Ritse
• Format: Article
• Language: English
• Subjects: Deep learning; Image synthesis; Magnetic Resonance Imaging; Multi-parametric fusion
• ISSN: 1566-2535; 1872-6305
• DOI: 10.1016/j.inffus.2024.102381

Magnetic resonance imaging (MRI) is highly sensitive for lesion detection. Sequences obtained with different settings can capture specific characteristics of lesions. Such multi-parametric MRI information has been shown to aid radiologist performance in lesion classification, as well as improving the performance of artificial intelligence models in various tasks. However, obtaining multi-parametric MRI makes the examination costly from both financial and time perspectives, and there may also be safety concerns for special populations, thus making acquisition of the full spectrum of MRI sequences less durable. In this study, a sophisticated Integrated MRI Multi-Parametric increment fusiOn generatoR wiTh AtteNTion Network (IMPORTANT-Net) is developed to generate absent sequences/parameters. First, the parameter reconstruction module is used to encode and restore the existing MRI parameters to obtain the corresponding latent representation information at any scale level. Then the multi-parametric fusion with attention module enables the interaction of the encoded information from different parameters through a set of algorithmic strategies, and applies different weights to the information through the attention mechanism after information fusion to obtain refined representation information. Finally, a increment fusion scheme embedded in a V-shape generation module is used to combine the hierarchical representations to generate specified absent MRI parameter. Results showed that our IMPORTANT-Net is capable of synthesizing absent MRI, outperforms comparable state-of-the-art networks and more importantly benefit downstream tasks. The codes are available at https://github.com/Netherlands-Cancer-Institute/MRI_IMPORTANT_NET •Multi-parameter fusion module realizes the interaction between different parameters.•The attention module optimizes the contribution of the fused parameters.•Hierarchical increment leads to both global and local analysis and fusion.•Synthesized images can improve the performance of AI models in downstream tasks.