Health equity assessment of machine learning performance (HEAL): a framework and dermatology AI model case study

• Published in: EClinicalMedicine 2024-04, Vol.70, p.102479-102479, Article 102479
• Main Authors: Schaekermann, Mike, Spitz, Terry, Pyles, Malcolm, Cole-Lewis, Heather, Wulczyn, Ellery, Pfohl, Stephen R., Martin, Donald, Jaroensri, Ronnachai, Keeling, Geoff, Liu, Yuan, Farquhar, Stephanie, Xue, Qinghan, Lester, Jenna, Hughes, Cían, Strachan, Patricia, Tan, Fraser, Bui, Peggy, Mermel, Craig H., Peng, Lily H., Matias, Yossi, Corrado, Greg S., Webster, Dale R., Virmani, Sunny, Semturs, Christopher, Liu, Yun, Horn, Ivor, Cameron Chen, Po-Hsuan
• Format: Article
• Language: English
• Subjects: Artificial intelligence; Dermatology; Health equity; Machine learning
• ISSN: 2589-5370; 2589-5370
• DOI: 10.1016/j.eclinm.2024.102479

Artificial intelligence (AI) has repeatedly been shown to encode historical inequities in healthcare. We aimed to develop a framework to quantitatively assess the performance equity of health AI technologies and to illustrate its utility via a case study. Here, we propose a methodology to assess whether health AI technologies prioritise performance for patient populations experiencing worse outcomes, that is complementary to existing fairness metrics. We developed the Health Equity Assessment of machine Learning performance (HEAL) framework designed to quantitatively assess the performance equity of health AI technologies via a four-step interdisciplinary process to understand and quantify domain-specific criteria, and the resulting HEAL metric. As an illustrative case study (analysis conducted between October 2022 and January 2023), we applied the HEAL framework to a dermatology AI model. A set of 5420 teledermatology cases (store-and-forward cases from patients of 20 years or older, submitted from primary care providers in the USA and skin cancer clinics in Australia), enriched for diversity in age, sex and race/ethnicity, was used to retrospectively evaluate the AI model's HEAL metric, defined as the likelihood that the AI model performs better for subpopulations with worse average health outcomes as compared to others. The likelihood that AI performance was anticorrelated to pre-existing health outcomes was estimated using bootstrap methods as the probability that the negated Spearman's rank correlation coefficient (i.e., “R”) was greater than zero. Positive values of R suggest that subpopulations with poorer health outcomes have better AI model performance. Thus, the HEAL metric, defined as p (R >0), measures how likely the AI technology is to prioritise performance for subpopulations with worse average health outcomes as compared to others (presented as a percentage below). Health outcomes were quantified as disability-adjusted life years (DALYs) when grouping by sex and age, and years of life lost (YLLs) when grouping by race/ethnicity. AI performance was measured as top-3 agreement with the reference diagnosis from a panel of 3 dermatologists per case. Across all dermatologic conditions, the HEAL metric was 80.5% for prioritizing AI performance of racial/ethnic subpopulations based on YLLs, and 92.1% and 0.0% respectively for prioritizing AI performance of sex and age subpopulations based on DALYs. Certain dermatologic conditions were significantly