SAFRON: Stitching Across the Frontier Network for Generating Colorectal Cancer Histology Images

• Published in: Medical image analysis 2022-04, Vol.77, p.102337-102337, Article 102337
• Main Authors: Deshpande, Srijay, Minhas, Fayyaz, Graham, Simon, Rajpoot, Nasir
• Format: Article
• Language: English
• Subjects: Algorithms; Annotated data generation; Cancer; Colorectal cancer; Colorectal carcinoma; Computational pathology; Computer applications; Deep learning; Generative adversarial networks; High resolution; Histology; Image quality; Image resolution; Image segmentation; Image synthesis; Machine learning; Mental task performance; Pathology; Realism; Stitching; Synthesis; Training
• ISSN: 1361-8415; 1361-8423
• DOI: 10.1016/j.media.2021.102337

•Model that can generate tissue tiles of large arbitrary sizes than the ones used for training using a novel stitching mechanism.•Computationally inexpensive and memory efficient framework.•Generation of colorectal tissue tiles of dimensions up to 11,000 × 8,000 pixels.•Ability to generate an unlimited amount of annotated synthetic data which can assist the gland segmentation and cancer detection algorithm. [Display omitted] Automated synthesis of histology images has several potential applications including the development of data-efficient deep learning algorithms. In the field of computational pathology, where histology images are large in size and visual context is crucial, synthesis of large high-resolution images via generative modeling is an important but challenging task due to memory and computational constraints. To address this challenge, we propose a novel framework called SAFRON (Stitching Across the FROntier Network) to construct realistic, large high-resolution tissue images conditioned on input tissue component masks. The main novelty in the framework is integration of stitching in its loss function which enables generation of images of arbitrarily large sizes after training on relatively small image patches while preserving morphological features with minimal boundary artifacts. We have used the proposed framework for generating, to the best of our knowledge, the largest-sized synthetic histology images to date (up to 11K×8K pixels). Compared to existing approaches, our framework is efficient in terms of the memory required for training and computations needed for synthesizing large high-resolution images. The quality of generated images was assessed quantitatively using Frechet Inception Distance as well as by 7 trained pathologists, who assigned a realism score to a set of images generated by SAFRON. The average realism score across all pathologists for synthetic images was as high as that of real images. We also show that training with additional synthetic data generated by SAFRON can significantly boost prediction performance of gland segmentation and cancer detection algorithms in colorectal cancer histology images.