Natural language processing for abstraction of cancer treatment toxicities: accuracy versus human experts

Objectives Expert abstraction of acute toxicities is critical in oncology research but is labor-intensive and variable. We assessed the accuracy of a natural language processing (NLP) pipeline to extract symptoms from clinical notes compared to physicians. Materials and Methods Two independent revie...

Full description

Saved in:
Bibliographic Details
Published in:JAMIA open 2020-12, Vol.3 (4), p.513-517
Main Authors: Hong, Julian C, Fairchild, Andrew T, Tanksley, Jarred P, Palta, Manisha, Tenenbaum, Jessica D
Format: Article
Language:English
Subjects:
Application Notes
Cancer
Care and treatment
Complications and side effects
Computational linguistics
Drug therapy
Fatigue
Language processing
Medical care
Natural language interfaces
Quality management
Radiation
Radiotherapy
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Objectives Expert abstraction of acute toxicities is critical in oncology research but is labor-intensive and variable. We assessed the accuracy of a natural language processing (NLP) pipeline to extract symptoms from clinical notes compared to physicians. Materials and Methods Two independent reviewers identified present and negated National Cancer Institute Common Terminology Criteria for Adverse Events (CTCAE) v5.0 symptoms from 100 randomly selected notes for on-treatment visits during radiation therapy with adjudication by a third reviewer. A NLP pipeline based on Apache clinical Text Analysis Knowledge Extraction System was developed and used to extract CTCAE terms. Accuracy was assessed by precision, recall, and F1. Results The NLP pipeline demonstrated high accuracy for common physician-abstracted symptoms, such as radiation dermatitis (F1 0.88), fatigue (0.85), and nausea (0.88). NLP had poor sensitivity for negated symptoms. Conclusion NLP accurately detects a subset of documented present CTCAE symptoms, though is limited for negated symptoms. It may facilitate strategies to more consistently identify toxicities during cancer therapy.
ISSN:2574-2531
2574-2531
DOI:10.1093/jamiaopen/ooaa064