RAPHIA: A deep learning pipeline for the registration of MRI and whole-mount histopathology images of the prostate

• Published in: Computers in biology and medicine 2024-05, Vol.173, p.108318, Article 108318
• Main Authors: Shao, Wei, Vesal, Sulaiman, Soerensen, Simon J.C., Bhattacharya, Indrani, Golestani, Negar, Yamashita, Rikiya, Kunder, Christian A., Fan, Richard E., Ghanouni, Pejman, Brooks, James D., Sonn, Geoffrey A., Rusu, Mirabela
• Format: Article
• Language: English
• Subjects: Computing time; Deep Learning; Estimation; Histopathology; Humans; Image Processing, Computer-Assisted - methods; Image registration; Learning algorithms; Machine learning; Magnetic resonance imaging; Magnetic Resonance Imaging - methods; Male; Medical imaging; MRI; Prostate - diagnostic imaging; Prostate cancer; Prostatic Neoplasms - diagnostic imaging; Radiology; Radiology-pathology fusion; Registration; Representations; Rotation
• ISSN: 0010-4825; 1879-0534; 1879-0534
• DOI: 10.1016/j.compbiomed.2024.108318

Image registration can map the ground truth extent of prostate cancer from histopathology images onto MRI, facilitating the development of machine learning methods for early prostate cancer detection. Here, we present RAdiology PatHology Image Alignment (RAPHIA), an end-to-end pipeline for efficient and accurate registration of MRI and histopathology images. RAPHIA automates several time-consuming manual steps in existing approaches including prostate segmentation, estimation of the rotation angle and horizontal flipping in histopathology images, and estimation of MRI-histopathology slice correspondences. By utilizing deep learning registration networks, RAPHIA substantially reduces computational time. Furthermore, RAPHIA obviates the need for a multimodal image similarity metric by transferring histopathology image representations to MRI image representations and vice versa. With the assistance of RAPHIA, novice users achieved expert-level performance, and their mean error in estimating histopathology rotation angle was reduced by 51% (12 degrees vs 8 degrees), their mean accuracy of estimating histopathology flipping was increased by 5% (95.3% vs 100%), and their mean error in estimating MRI-histopathology slice correspondences was reduced by 45% (1.12 slices vs 0.62 slices). When compared to a recent conventional registration approach and a deep learning registration approach, RAPHIA achieved better mapping of histopathology cancer labels, with an improved mean Dice coefficient of cancer regions outlined on MRI and the deformed histopathology (0.44 vs 0.48 vs 0.50), and a reduced mean per-case processing time (51 vs 11 vs 4.5 min). The improved performance by RAPHIA allows efficient processing of large datasets for the development of machine learning models for prostate cancer detection on MRI. Our code is publicly available at: https://github.com/pimed/RAPHIA. [Display omitted] •RAPHIA is a fully automated prostate MRI-histopathology image registration pipeline.•RAPHIA avoids the need for a multimodal similarity metric through image translation.•RAPHIA is user-friendly and can help novice users achieve expert-level performance.•RAPHIA is a human-in-the-loop pipeline that allows intervention by advanced users.