Deep learning for whole-body medical image generation

• Published in: European journal of nuclear medicine and molecular imaging 2021-11, Vol.48 (12), p.3817-3826
• Main Authors: Schaefferkoetter, Joshua, Yan, Jianhua, Moon, Sangkyu, Chan, Rosanna, Ortega, Claudia, Metser, Ur, Berlin, Alejandro, Veit-Haibach, Patrick
• Format: Article
• Language: English
• Subjects: Advanced Image Analyses (Radiomics and Artificial Intelligence); Algorithms; Artificial Intelligence; Attenuation; Cardiology; Computed tomography; Data acquisition; Deep Learning; Generative adversarial networks; Human Body; Humans; Image processing; Image Processing, Computer-Assisted; Image quality; Imaging; Machine learning; Magnetic Resonance Imaging; Medical imaging; Medicine; Medicine & Public Health; Multimodal Imaging; Nuclear Medicine; Oncology; Original Article; Orthopedics; Patients; Performance evaluation; Positron emission; Positron Emission Tomography Computed Tomography; Positron-Emission Tomography; Radiology; Tomography; Tomography, X-Ray Computed; Training; Transformations (mathematics)
• ISSN: 1619-7070; 1619-7089
• DOI: 10.1007/s00259-021-05413-0

Background Artificial intelligence (AI) algorithms based on deep convolutional networks have demonstrated remarkable success for image transformation tasks. State-of-the-art results have been achieved by generative adversarial networks (GANs) and training approaches which do not require paired data. Recently, these techniques have been applied in the medical field for cross-domain image translation. Purpose This study investigated deep learning transformation in medical imaging. It was motivated to identify generalizable methods which would satisfy the simultaneous requirements of quality and anatomical accuracy across the entire human body. Specifically, whole-body MR patient data acquired on a PET/MR system were used to generate synthetic CT image volumes. The capacity of these synthetic CT data for use in PET attenuation correction (AC) was evaluated and compared to current MR-based attenuation correction (MR-AC) methods, which typically use multiphase Dixon sequences to segment various tissue types. Materials and methods This work aimed to investigate the technical performance of a GAN system for general MR-to-CT volumetric transformation and to evaluate the performance of the generated images for PET AC. A dataset comprising matched, same-day PET/MR and PET/CT patient scans was used for validation. Results A combination of training techniques was used to produce synthetic images which were of high-quality and anatomically accurate. Higher correlation was found between the values of mu maps calculated directly from CT data and those derived from the synthetic CT images than those from the default segmented Dixon approach. Over the entire body, the total amounts of reconstructed PET activities were similar between the two MR-AC methods, but the synthetic CT method yielded higher accuracy for quantifying the tracer uptake in specific regions. Conclusion The findings reported here demonstrate the feasibility of this technique and its potential to improve certain aspects of attenuation correction for PET/MR systems. Moreover, this work may have larger implications for establishing generalized methods for inter-modality, whole-body transformation in medical imaging. Unsupervised deep learning techniques can produce high-quality synthetic images, but additional constraints may be needed to maintain medical integrity in the generated data. Graphical abstract