Diagnosis of osteosarcoma based on multimodal microscopic imaging and deep learning

• Published in: Journal of innovative optical health science 2024-02
• Main Authors: Wang, Zihan, Wu, Jinjin, Li, Chenbei, Wang, Bing, Wu, Qingxia, Li, Lan, Wang, Huijie, Tu, Chao, Yin, Jianhua
• Format: Article
• Language: English
• ISSN: 1793-5458; 1793-7205
• DOI: 10.1142/S1793545823430010

Osteosarcoma is the most common primary bone tumor with high malignancy. It is particularly necessary to achieve rapid and accurate diagnosis in its intraoperative examination and early diagnosis. Accordingly, the multimodal microscopic imaging diagnosis system constructed by bright field, spontaneous fluorescence and polarized light microscopic imaging was used to study the pathological mechanism of osteosarcoma from the tissue microenvironment level and achieve rapid and accurate diagnosis. First, the multimodal microscopic images of normal and osteosarcoma tissue slices were collected to characterize the overall morphology of the tissue microenvironment of the samples, the arrangement structure of collagen fibers and the content and distribution of endogenous fluorescent substances. Second, based on the correlation and complementarity of the feature information contained in the three single-mode images, combined with convolutional neural network (CNN) and image fusion methods, a multimodal intelligent diagnosis model was constructed to effectively improve the information utilization and diagnosis accuracy. The accuracy and true positivity of the multimodal diagnostic model were significantly improved to 0.8495 and 0.9412, respectively, compared to those of the single-modal models. Besides, the difference of tissue microenvironments before and after cancerization can be used as a basis for cancer diagnosis, and the information extraction and intelligent diagnosis of osteosarcoma tissue can be achieved by using multimodal microscopic imaging technology combined with deep learning, which significantly promoted the application of tissue microenvironment in pathological examination. This diagnostic system relies on its advantages of simple operation, high efficiency and accuracy and high cost-effectiveness, and has enormous clinical application potential and research significance.