Model Compression via Pattern Shared Sparsification in Analog Federated Learning under Communication Constraints

Recently, it has been shown that analog transmission based federated learning enables more efficient usage of communication resources compared to the conventional digital transmission. In this paper, we propose an effective model compression strategy enabling analog FL under constrained communicatio...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on green communications and networking 2023-03, Vol.7 (1), p.1-1
Main Authors: Ahn, Jin-Hyun, Bennis, Mehdi, Kang, Joonhyuk
Format: Article
Language:English
Subjects:
Bandwidth
Bandwidths
Budgets
Closed form solutions
Collaborative work
Communication
compression
Constraints
Convergence
Digital transmission
Distributed learning
Exact solutions
Federated learning
local gradient accumulation
Mathematical models
Numerical models
over-the-air computation
Parameters
Performance enhancement
Performance evaluation
Uplink
Wireless communication
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Recently, it has been shown that analog transmission based federated learning enables more efficient usage of communication resources compared to the conventional digital transmission. In this paper, we propose an effective model compression strategy enabling analog FL under constrained communication bandwidth. To this end, the proposed approach is based on pattern shared sparsification by setting the same sparsification pattern of parameter vectors uploaded by edge devices, as opposed to each edge device independently applying sparsification. In particular, we propose specific schemes for determining the sparsification pattern and characterize the convergence of analog FL leveraging these proposed sparsification strategies, by deriving a closed-form upper bound of convergence rate and residual error. The closed-form expression allows to capture the effect of communication bandwidth and power budget to the performance of analog FL. In terms of convergence analysis, the model parameter obtained with the proposed schemes is proven to converge to the optimum of model parameter. Numerical results show that leveraging the proposed pattern shared sparsification consistently improves the performance of analog FL in various settings of system parameters. The improvement in performance is more significant under scarce communication bandwidth and limited transmit power budget.
ISSN:2473-2400
2473-2400
DOI:10.1109/TGCN.2022.3186538