Ensemble learning for poor prognosis predictions: A case study on SARS-CoV-2

• Published in: Journal of the American Medical Informatics Association : JAMIA 2021-03, Vol.28 (4), p.791-800
• Main Authors: Wu, Honghan, Zhang, Huayu, Karwath, Andreas, Ibrahim, Zina, Shi, Ting, Zhang, Xin, Wang, Kun, Sun, Jiaxing, Dhaliwal, Kevin, Bean, Daniel, Cardoso, Victor Roth, Li, Kezhi, Teo, James T, Banerjee, Amitava, Gao-Smith, Fang, Whitehouse, Tony, Veenith, Tonny, Gkoutos, Georgios V, Wu, Xiaodong, Dobson, Richard, Guthrie, Bruce
• Format: Article
• Language: English
• Subjects: Adult; Aged; Aged, 80 and over; China - epidemiology; COVID-19 - epidemiology; COVID-19 - mortality; COVID-19 - prevention & control; Female; Humans; Male; Middle Aged; Models, Statistical; Prognosis; Research and Applications; Risk Assessment - methods; SARS-CoV-2; United Kingdom - epidemiology
• ISSN: 1527-974X; 1067-5027; 1527-974X
• DOI: 10.1093/jamia/ocaa295

Risk prediction models are widely used to inform evidence-based clinical decision making. However, few models developed from single cohorts can perform consistently well at population level where diverse prognoses exist (such as the SARS-CoV-2 [severe acute respiratory syndrome coronavirus 2] pandemic). This study aims at tackling this challenge by synergizing prediction models from the literature using ensemble learning. In this study, we selected and reimplemented 7 prediction models for COVID-19 (coronavirus disease 2019) that were derived from diverse cohorts and used different implementation techniques. A novel ensemble learning framework was proposed to synergize them for realizing personalized predictions for individual patients. Four diverse international cohorts (2 from the United Kingdom and 2 from China; N = 5394) were used to validate all 8 models on discrimination, calibration, and clinical usefulness. Results showed that individual prediction models could perform well on some cohorts while poorly on others. Conversely, the ensemble model achieved the best performances consistently on all metrics quantifying discrimination, calibration, and clinical usefulness. Performance disparities were observed in cohorts from the 2 countries: all models achieved better performances on the China cohorts. When individual models were learned from complementary cohorts, the synergized model had the potential to achieve better performances than any individual model. Results indicate that blood parameters and physiological measurements might have better predictive powers when collected early, which remains to be confirmed by further studies. Combining a diverse set of individual prediction models, the ensemble method can synergize a robust and well-performing model by choosing the most competent ones for individual patients.