Reducing segmentation failures in cardiac MRI via late feature fusion and GAN-based augmentation

• Published in: Computers in biology and medicine 2023-07, Vol.161, p.106973-106973, Article 106973
• Main Authors: Al Khalil, Yasmina, Amirrajab, Sina, Lorenz, Cristian, Weese, Jürgen, Pluim, Josien, Breeuwer, Marcel
• Format: Article
• Language: English
• Subjects: Algorithms; Cardiac function; Cardiac segmentation; Cardiovascular disease; Deep learning; GANs; Heart - diagnostic imaging; Heart Diseases; Humans; Image processing; Image Processing, Computer-Assisted - methods; Image segmentation; Image synthesis; Late fusion; Magnetic resonance; Magnetic resonance imaging; Magnetic Resonance Imaging - methods; Medical imaging; Outlier failure reduction; Outliers (statistics); Radiography; Training
• ISSN: 0010-4825; 1879-0534
• DOI: 10.1016/j.compbiomed.2023.106973

Cardiac magnetic resonance (CMR) image segmentation is an integral step in the analysis of cardiac function and diagnosis of heart related diseases. While recent deep learning-based approaches in automatic segmentation have shown great promise to alleviate the need for manual segmentation, most of these are not applicable to realistic clinical scenarios. This is largely due to training on mainly homogeneous datasets, without variation in acquisition, which typically occurs in multi-vendor and multi-site settings, as well as pathological data. Such approaches frequently exhibit a degradation in prediction performance, particularly on outlier cases commonly associated with difficult pathologies, artifacts and extensive changes in tissue shape and appearance. In this work, we present a model aimed at segmenting all three cardiac structures in a multi-center, multi-disease and multi-view scenario. We propose a pipeline, addressing different challenges with segmentation of such heterogeneous data, consisting of heart region detection, augmentation through image synthesis and a late-fusion segmentation approach. Extensive experiments and analysis demonstrate the ability of the proposed approach to tackle the presence of outlier cases during both training and testing, allowing for better adaptation to unseen and difficult examples. Overall, we show that the effective reduction of segmentation failures on outlier cases has a positive impact on not only the average segmentation performance, but also on the estimation of clinical parameters, leading to a better consistency in derived metrics. •An optimized framework for segmentation of heterogeneous cardiac MR images.•Method includes heart detection, image synthesis, and late-fusion segmentation.•Conditional GAN to augment training with realistically synthesized pathology images.•Multi-path late fusion combined with intensity transforms for improved robustness.•Segmentation failure reduction improves consistency in derived clinical metrics.