Synthetic Histopathological Images Generation with Artificial Intelligence to Accelerate Research and Improve Clinical Outcomes in Hematology

• Published in: Blood 2023-11, Vol.142 (Supplement 1), p.902-902
• Main Authors: Asti, Gianluca, D'Amico, Saverio, Curti, Nico, Carlini, Gianluca, Sauta, Elisabetta, Derus, Nicolas Riccardo, Dall'Olio, Daniele, Sala, Claudia, Dall'Olio, Lorenzo, Lanino, Luca, Maggioni, Giulia, Campagna, Alessia, Ubezio, Marta, Russo, Antonio, Todisco, Gabriele, Tentori, Cristina Astrid, Morandini, Pierandrea, Bicchieri, Marilena, Grondelli, Maria Chiara, Zampini, Matteo, Savevski, Victor, Santoro, Armando, Kordasti, Shahram, Santini, Valeria, Kubasch, Anne Sophie, Platzbecker, Uwe, Diez-Campelo, Maria, Fenaux, Pierre, Zhao, Lin-Pierre, Zeidan, Amer M., Haferlach, Torsten, Castellani, Gastone, Della Porta, Matteo Giovanni
• Format: Article
• Language: English
• ISSN: 0006-4971; 1528-0020
• DOI: 10.1182/blood-2023-187521

Background: Hematological malignancies are rare and complex diseases and as a consequence, multimodal data (ranging from clinical and genomic information to images) are required to improve diagnosis, prognosis and personalized treatments. However, collecting all these layers of information is challenging, in particular when collecting cytological and histological images from the bone marrow (BM) reproducing disease morphologic features. Synthetic data generation by Artificial Intelligence (AI) can circumvent these issues by generating images conditioned from textual inputs (i.e. reports from pathologists), which are widely available and contain many useful clinical information. This technology can enrich data with synthetic images, thus boosting translational research and improving the performances of precision medicine strategies based on multimodal information. Aims:This project was conducted by GenoMed4all and Synthema EU consortia, with the aim to:1)Apply generative models to real-world dataset with histological images of patients with myeloid neoplasms (MN). 2) Develop a Synthetic Images Validation Framework (SIVF) to evaluate the utility and fidelity of generated images. 3) Verify the capability of synthetic images to accelerate research and to improve clinical models. Methods:We implemented Stable Diffusion (SD) generative model fine-tuned on hematological data to generate Hematoxylin and Eosin (H&E) images of MN patients. We implemented a domain specific language model (HematoBERT) to encode textual input as condition for the generation process. Use cases were Myelodysplastic Syndrome (MDS), Acute Myeloid Leukemia (AML) and Myeloproliferative Neoplasm (MPN) patients, with available BM biopsies and their reports from pathologists, genomic and clinical data. We applied SIVF to evaluate distributions of morphological features extracted from real and synthetic images. Clinical validation was performed on disease classification and survival probability prediction, using real and synthetic images features (experimental setting is reported in Figure 1). Results: We trained SD model on 200 patients with available BM biopsies and associated reports. We first performed SIVF to compare extracted morphological features (geometrical, color and texture features of cells nuclei) from synthetic and real images of 55 patients never seen by the model. Results proved that features distributions and correlations in both datasets were comparable. Similar results were o