Evaluation of federated learning variations for COVID-19 diagnosis using chest radiographs from 42 US and European hospitals

Federated learning (FL) allows multiple distributed data holders to collaboratively learn a shared model without data sharing. However, individual health system data are heterogeneous. "Personalized" FL variations have been developed to counter data heterogeneity, but few have been evaluat...

Full description

Saved in:
Bibliographic Details
Published in:Journal of the American Medical Informatics Association : JAMIA 2022-12, Vol.30 (1), p.54-63
Main Authors: Peng, Le, Luo, Gaoxiang, Walker, Andrew, Zaiman, Zachary, Jones, Emma K, Gupta, Hemant, Kersten, Kristopher, Burns, John L, Harle, Christopher A, Magoc, Tanja, Shickel, Benjamin, Steenburg, Scott D, Loftus, Tyler, Melton, Genevieve B, Gichoya, Judy Wawira, Sun, Ju, Tignanelli, Christopher J
Format: Article
Language:English
Subjects:
COVID-19
COVID-19 Testing
Europe
Hospitals
Humans
Learning
United States
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Federated learning (FL) allows multiple distributed data holders to collaboratively learn a shared model without data sharing. However, individual health system data are heterogeneous. "Personalized" FL variations have been developed to counter data heterogeneity, but few have been evaluated using real-world healthcare data. The purpose of this study is to investigate the performance of a single-site versus a 3-client federated model using a previously described Coronavirus Disease 19 (COVID-19) diagnostic model. Additionally, to investigate the effect of system heterogeneity, we evaluate the performance of 4 FL variations. We leverage a FL healthcare collaborative including data from 5 international healthcare systems (US and Europe) encompassing 42 hospitals. We implemented a COVID-19 computer vision diagnosis system using the Federated Averaging (FedAvg) algorithm implemented on Clara Train SDK 4.0. To study the effect of data heterogeneity, training data was pooled from 3 systems locally and federation was simulated. We compared a centralized/pooled model, versus FedAvg, and 3 personalized FL variations (FedProx, FedBN, and FedAMP). We observed comparable model performance with respect to internal validation (local model: AUROC 0.94 vs FedAvg: 0.95, P = .5) and improved model generalizability with the FedAvg model (P 
ISSN:1067-5027
1527-974X
DOI:10.1093/jamia/ocac188