CT and MRI fusion based on generative adversarial network and convolutional neural networks under image enhancement

• Published in: Sheng wu yi xue gong cheng xue za zhi 2023-04, Vol.40 (2), p.208-216
• Main Authors: Liu, Yunpeng, Li, Jin, Wang, Yu, Cai, Wenli, Chen, Fei, Liu, Wenjie, Mao, Xianhao, Gan, Kaifeng, Wang, Renfang, Sun, Dechao, Qiu, Hong, Liu, Bangquan
• Format: Article
• Language: Chinese
• Subjects: Algorithms; Artificial neural networks; Computed tomography; Computer vision; Entropy (Information theory); Frequency dependence; Generative adversarial networks; Image enhancement; Image Processing, Computer-Assisted - methods; Magnetic resonance imaging; Magnetic Resonance Imaging - methods; Medical imaging; Neural networks; Neural Networks, Computer; Tomography, X-Ray Computed; Transfer learning
• ISSN: 1001-5515
• DOI: 10.7507/1001-5515.202209050

Aiming at the problems of missing important features, inconspicuous details and unclear textures in the fusion of multimodal medical images, this paper proposes a method of computed tomography (CT) image and magnetic resonance imaging (MRI) image fusion using generative adversarial network (GAN) and convolutional neural network (CNN) under image enhancement. The generator aimed at high-frequency feature images and used double discriminators to target the fusion images after inverse transform; Then high-frequency feature images were fused by trained GAN model, and low-frequency feature images were fused by CNN pre-training model based on transfer learning. Experimental results showed that, compared with the current advanced fusion algorithm, the proposed method had more abundant texture details and clearer contour edge information in subjective representation. In the evaluation of objective indicators, Q , information entropy (IE), spatial frequency (SF), structural similarity (SSIM), mutual information (MI) a