Communication-Efficient Federated Multitask Learning Over Wireless Networks

• Published in: IEEE internet of things journal 2023-01, Vol.10 (1), p.609-624
• Main Authors: Ma, Haoyu, Guo, Huayan, Lau, Vincent K. N.
• Format: Article
• Language: English
• Subjects: Algorithms; Artificial neural networks; Communication; Convergence; Data transfer (computers); Dynamic scheduling; Feature extraction; Federated multitask learning (FMTL); Learning; Lyapunov analysis; Modules; Network latency; Neural networks; Scheduling; Task analysis; Task scheduling; Training; user scheduling; Wireless communications; wireless federated learning (FL); Wireless networks
• ISSN: 2327-4662; 2327-4662
• DOI: 10.1109/JIOT.2022.3201310

This article investigates the scheduling framework of the federated multitask learning (FMTL) problem with a hard-cooperation structure over wireless networks, in which the scheduling becomes more challenging due to the different convergence behaviors of different tasks. Based on the special model structure, we propose a dynamic user and task scheduling scheme with a block-wise incremental gradient aggregation algorithm, in which the neural network model is decomposed into a common feature-extraction module and M task-specific modules. Different block gradients with respect to different modules can be scheduled separately. We further propose a Lyapunov-drift-based scheduling scheme that minimizes the overall communication latency by utilizing both the instantaneous data importance and the channel state information. We prove that the proposed scheme can converge almost surely to a KKT solution of the training problem such that the data-distortion issue is resolved. Simulation results illustrate that the proposed scheme significantly reduces the communication latency compared to the state-of-the-art baseline schemes.