Compressing Model before Federated Learning by Transferrable Surrogate Lottery Ticket

Federated learning (FL) is a promising technique to collaboratively train a model with distributed users and datasets. To develop communication-efficient FL systems, model-size reduction by using a winning ticket of the lottery ticket hypothesis has been proposed and investigated; however, it is sti...

Full description

Saved in:
Bibliographic Details
Main Authors: Tanimura, Takahito, Takase, Masayuki
Format: Conference Proceeding
Language:English
Subjects:
Costs
Federated learning
Lottery Ticket Hypothesis
Machine Learning
Model Pruning
Performance evaluation
Servers
Task analysis
Training
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Federated learning (FL) is a promising technique to collaboratively train a model with distributed users and datasets. To develop communication-efficient FL systems, model-size reduction by using a winning ticket of the lottery ticket hypothesis has been proposed and investigated; however, it is still an issue how to discover winning tickets in FL systems, which incur communication costs for model training to find the winning ticket. To address this issue, we propose a method of using a surrogate dataset that can be synthetically generated at an FL server, instead of the distributed target datasets found at the clients, for finding the winning ticket. The method is based on observations that winning tickets obtained from a large dataset could be transferable to other tasks. The performance evaluations using a subset of the FEMNIST (as target) and Chars 74K (as emulated surrogate) datasets show that the proposed method reduces communication cost in FL by about 80%.
ISSN:2331-9860
DOI:10.1109/CCNC51644.2023.10060578