Machine Learning Approach to Estimating ECOG PS for a Multiple-Myeloma Cohort from Real World Data

• Published in: Blood 2023-11, Vol.142 (Supplement 1), p.4700-4700
• Main Authors: Graham, Steven, Graham, Jeff, O'Rourke, Julia, Deveras, Claudia, Theocharous, Panteli, Rose, Virgil, Pelloso, Luis Arthur F., Sudarsanam, Sucha
• Format: Article
• Language: English
• ISSN: 0006-4971; 1528-0020
• DOI: 10.1182/blood-2023-182252

Background: ECOG Performance Status Scale (ECOG PS) is a measure of patient's ability to function, and an important clinical attribute used in treatment decisions and trial recruitment. In the real world, therapies might be used for patients with more advanced disease, lower functional status, and a higher dependency on others for daily activities, and this population is often excluded from registry trials.This is particularly important for multiple myeloma (MM) in which patients often present with advanced disease and require new therapies despite their high ECOG scores.Of note, in real-world data sources (RWD), ECOG PS is often not recorded by physicians in either structured or unstructured clinical notes. To better understand patient and physician preferences amongst the new treatments available and to generate insights into distinct clinical subgroups, we developed a machine learning (ML) method to estimate ECOG PS for a multiple myeloma cohort using a curated RWD data set. Methods: A curated RWD data set of 126K patients diagnosed with MM was derived from a federated Electronic Health Record network of 57 Healthcare Organizations in the US. Natural Language Processing (NLP) methods were used to extract ECOG scores from clinical notes for 4,799 patients and used as ground truth for model building. Based on input from clinical experts, we identified 96 features including MM specific features such as disease characteristics including, ISS Staging, CRAB values, etc. as well as features indicative of patient performance such as use of orthotic device, stroke/brain injury, days in inpatient settings, etc. Our initial experiments suggested that non-MM specific variables may contribute significantly to the performance of the model. For our final model building we included 38 non-MM specific features after subtracting features of low variance and collinearity. Dataset was divided into training, testing and validation sets with a split of 60, 20, 20 respectively. We used an XGBoost algorithm and tuned hyper-parameters for the classifier. We tested both multiclass (ECOG PS 0-4) and binary (0-1, 2-4) classifiers. The resulting model was used to estimate ECOG scores longitudinally, first at time of diagnosis and subsequently on a date which meets all of the following: at least one year after diagnosis, had key features available and prior to discontinuation of MM treatments. Succeeding ECOG scores were estimated at least 365 days apart to ensure patients were moni