Auxiliary diagnosis study of integrated electronic medical record text and CT images

• Published in: Journal of intelligent systems 2022-06, Vol.31 (1), p.753-766
• Main Authors: Yuanchuan, Duan, Hang, Diao, Shi, Li, Kailin, Liu, Yijie, Feng
• Format: Article
• Language: English
• Subjects: Accuracy; Bi-LSTM model; Computed tomography; Diagnosis; Electronic health records; Feature extraction; fusion features; GoogleNet model; Medical imaging; Medical records; Medical research; medical-assisted diagnosis
• ISSN: 2191-026X; 0334-1860; 2191-026X
• DOI: 10.1515/jisys-2022-0040

At present, most of the research in the field of medical-assisted diagnosis is carried out based on image or electronic medical records. Although there is some research foundation, they lack the comprehensive consideration of comprehensive image and text modes. Based on this situation, this article proposes a fusion classification auxiliary diagnosis model based on GoogleNet model and Bi-LSTM model, uses GoogleNet to process brain computed tomographic (CT) images of ischemic stroke patients and extract CT image features, uses Bi-LSTM model to extract the electronic medical record text, integrates the two features using the full connection layer network and Softmax classifier, and obtains a method that can assist the diagnosis from two modes. Experiments show that the proposed scheme on average improves 3.05% in accuracy compared to individual image or text modes, and the best performing GoogleNet + Bi-LSTM model achieves 96.61% accuracy; although slightly less in recall, it performs better on 1 values, and has provided feasible new ideas and new methods for research in the field of multi-model medical-assisted diagnosis.