WAFFLe: Weight Anonymized Factorization for Federated Learning

In domains where data are sensitive or private, there is great value in methods that can learn in a distributed manner without the data ever leaving the local devices. In light of this need, federated learning has emerged as a popular training paradigm. However, many federated learning approaches tr...

Full description

Saved in:
Bibliographic Details
Published in:IEEE access 2022, Vol.10, p.49207-49218
Main Authors: Hao, Weituo, Mehta, Nikhil, Liang, Kevin J., Cheng, Pengyu, El-Khamy, Mostafa, Carin, Lawrence
Format: Article
Language:English
Subjects:
Collaborative work
Communication
Data models
Dictionaries
Distributed databases
Factorization
Federated learning
Heterogeneity
Indian buffet process
Neural networks
Performance degradation
Performance evaluation
personalization and fairness
Servers
Training
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In domains where data are sensitive or private, there is great value in methods that can learn in a distributed manner without the data ever leaving the local devices. In light of this need, federated learning has emerged as a popular training paradigm. However, many federated learning approaches trade transmitting data for communicating updated weight parameters for each local device. Therefore, a successful breach that would have otherwise directly compromised the data instead grants whitebox access to the local model, which opens the door to a number of attacks, including exposing the very data federated learning seeks to protect. Additionally, in distributed scenarios, individual client devices commonly exhibit high statistical heterogeneity. Many common federated approaches learn a single global model; while this may do well on average, performance degrades when the i.i.d. assumption is violated, underfitting individuals further from the mean and raising questions of fairness. To address these issues, we propose Weight Anonymized Factorization for Federated Learning (WAFFLe), an approach that combines the Indian Buffet Process with a shared dictionary of weight factors for neural networks. Experiments on MNIST, FashionMNIST, and CIFAR-10 demonstrate WAFFLe's significant improvement to local test performance and fairness while simultaneously providing an extra layer of security.
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2022.3172945