A comparison of natural language processing to ICD-10 codes for identification and characterization of pulmonary embolism

• Published in: Thrombosis research 2021-07, Vol.203, p.190-195
• Main Authors: Johnson, Stacy A., Signor, Emily A., Lappe, Katie L., Shi, Jianlin, Jenkins, Stephen L., Wikstrom, Sara W., Kroencke, Rachel D., Hallowell, David, Jones, Aubrey E., Witt, Daniel M.
• Format: Article
• Language: English
• Subjects: Accuracy; ICD-10; Natural language processing; NLP; Pulmonary embolism
• ISSN: 0049-3848; 1879-2472
• DOI: 10.1016/j.thromres.2021.04.020

The 10th revision of the International Classification of Diseases (ICD-10) codes is frequently used to identify pulmonary embolism (PE) events, although the validity of ICD-10 has been questioned. Natural language processing (NLP) is a novel tool that may be useful for pulmonary embolism identification. We performed a retrospective comparative accuracy study of 1000 randomly selected healthcare encounters with a CT pulmonary angiogram ordered between January 1, 2019 and January 1, 2020 at a single academic medical center. Two independent observers reviewed each radiology report and abstracted key findings related to PE presence/absence, chronicity, and anatomic location. NLP interpretations of radiology reports and ICD-10 codes were queried electronically and compared to the reference standard, manual chart review. A total of 970 encounters were included for analysis. The prevalence of PE was 13% by manual review. For PE identification, sensitivity was similar between NLP (96.0%) and ICD-10 (92.9%; p = 0.405), and specificity was significantly higher with NLP (97.7%) compared to ICD-10 (91.0%; p