Enhancing Global Model Performance in Federated Learning With Non-IID Data Using a Data-Free Generative Diffusion Model

Federated Learning (FL) presents a decentralized approach to machine learning, allowing multiple clients to jointly train neural networks while maintaining the privacy of their local data. However, FL faces challenges due to data heterogeneity, leading to slow convergence and reduced performance. Wh...

Full description

Saved in:
Bibliographic Details
Published in:IEEE access 2024, Vol.12, p.148230-148239
Main Authors: Najafi, Mohammadreza, Daneshtalab, Masoud, Lee, Jeong-A, Saadloonia, Ghazal, Shin, Seokjoo
Format: Article
Language:English
Subjects:
Accuracy
Clients
Computational modeling
Data models
Data privacy
Data transmission
Deep learning
diffusion model
Diffusion models
Diffusion rate
Federated learning
Generators
Heterogeneity
Knowledge discovery
knowledge distillation
Knowledge management
Machine learning
Mathematical models
Neural networks
Noise measurement
non-identical and independent distributions (Non-IID)
Privacy
Servers
Training
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Federated Learning (FL) presents a decentralized approach to machine learning, allowing multiple clients to jointly train neural networks while maintaining the privacy of their local data. However, FL faces challenges due to data heterogeneity, leading to slow convergence and reduced performance. While sharing client information can mitigate data heterogeneity, it poses a dilemma between privacy preservation and model performance improvement. This study aims to tackle the challenge of data heterogeneity, particularly for Non-Identical and Independent Distributions (Non-IID) clients, by enhancing the global model. We propose a data-free knowledge distillation method (FedDiff) to fine-tune the global model on the server. FedDiff leverages a Diffusion model as a generator to explore the input space of local models and transfer knowledge from local models to the global one. Additionally, we customize the diffusion model's data generative scheme to reduce training time. Extensive experiments demonstrate that FedDiff reduces communication rounds between clients and the server by up to 57% for CIFAR-10 classification and up to 71% for CIFAR-100 on average, compared to other state-of-the-art FL methods, while maintaining the same level of accuracy. This makes it particularly suitable for low-power devices with limitations on data transmission and reception, such as satellites and medical care devices.Furthermore, it preserves better average accuracy for all clients at the end of the training phase.
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2024.3474056