Performance of a Machine Learning Algorithm Using Electronic Health Record Data to Predict Postoperative Complications and Report on a Mobile Platform

• Published in: JAMA network open 2022-05, Vol.5 (5), p.e2211973-e2211973
• Main Authors: Ren, Yuanfang, Loftus, Tyler J, Datta, Shounak, Ruppert, Matthew M, Guan, Ziyuan, Miao, Shunshun, Shickel, Benjamin, Feng, Zheng, Giordano, Chris, Upchurch, Jr, Gilbert R, Rashidi, Parisa, Ozrazgat-Baslanti, Tezcan, Bihorac, Azra
• Format: Article
• Language: English
• Subjects: Adult; Algorithms; Artificial Intelligence; Automation; Electronic Health Records; Female; Health Informatics; Humans; Machine Learning; Male; Middle Aged; Missing data; Mortality; Online Only; Original Investigation; Postoperative Complications - epidemiology; Postoperative Complications - etiology; Prospective Studies; Retrospective Studies; Web portals
• ISSN: 2574-3805; 2574-3805
• DOI: 10.1001/jamanetworkopen.2022.11973

Predicting postoperative complications has the potential to inform shared decisions regarding the appropriateness of surgical procedures, targeted risk-reduction strategies, and postoperative resource use. Realizing these advantages requires that accurate real-time predictions be integrated with clinical and digital workflows; artificial intelligence predictive analytic platforms using automated electronic health record (EHR) data inputs offer an intriguing possibility for achieving this, but there is a lack of high-level evidence from prospective studies supporting their use. To examine whether the MySurgeryRisk artificial intelligence system has stable predictive performance between development and prospective validation phases and whether it is feasible to provide automated outputs directly to surgeons' mobile devices. In this prognostic study, the platform used automated EHR data inputs and machine learning algorithms to predict postoperative complications and provide predictions to surgeons, previously through a web portal and currently through a mobile device application. All patients 18 years or older who were admitted for any type of inpatient surgical procedure (74 417 total procedures involving 58 236 patients) between June 1, 2014, and September 20, 2020, were included. Models were developed using retrospective data from 52 117 inpatient surgical procedures performed between June 1, 2014, and November 27, 2018. Validation was performed using data from 22 300 inpatient surgical procedures collected prospectively from November 28, 2018, to September 20, 2020. Algorithms for generalized additive models and random forest models were developed and validated using real-time EHR data. Model predictive performance was evaluated primarily using area under the receiver operating characteristic curve (AUROC) values. Among 58 236 total adult patients who received 74 417 major inpatient surgical procedures, the mean (SD) age was 57 (17) years; 29 226 patients (50.2%) were male. Results reported in this article focus primarily on the validation cohort. The validation cohort included 22 300 inpatient surgical procedures involving 19 132 patients (mean [SD] age, 58 [17] years; 9672 [50.6%] male). A total of 2765 patients (14.5%) were Black or African American, 14 777 (77.2%) were White, 1235 (6.5%) were of other races (including American Indian or Alaska Native, Asian, Native Hawaiian or Pacific Islander, and multiracial), and 355 (1.9%) were of unknown race be