Differentially Private Online Federated Learning With Personalization and Fairness

• Main Author: Odeyomi, Olusola T.
• Format: Conference Proceeding
• Language: English
• Subjects: Algorithms; Data models; Differential privacy; fairness; Federated learning; Lagrangian optimization; meta-learning; Metalearning; online learning; Servers; Training
• ISSN: 2157-8117
• DOI: 10.1109/ISIT54713.2023.10206945

State-of-the-art federated learning algorithms are faced with many critical challenges that limit their applications in many domains. The vanilla federated learning architecture contains a central server that coordinates the training process. However, the central server is prone to congestion and a single point of failure as the number of clients grows exponentially. More so, the state-of-the-art federated learning algorithms are designed for fixed data distributions. Yet in many real-world scenarios, such as in real-time traffic monitoring systems, the data distribution of each client is time-varying. Hence, such algorithms do not perform well in these scenarios. Another challenge is that a single trained global model in federated learning may not perform optimally on the test dataset of any client since each client may have a specific task it aims to achieve despite task relatedness among clients. A more serious challenge caused by high data heterogeneity among the clients' dataset is the unfairness of the trained model towards a minority group. Minority groups are classified by race, gender, social status, etc. This unfairness occurs when the trained global model discriminates against minority groups even when a local model is bias-free. Therefore, this paper addresses these challenges by proposing the Differentially private Personalized Online Federated Learning (DIPOFEL) and Differentially private Personalized Online Federated Learning with FAIRness (DIPOFLAIR) Algorithms for a peer-to-peer federated learning setting reinforced with local differential privacy. DIPOFEL Algorithm employs the online learning technique to handle time-varying data distribution, the meta-learning technique to personalize the trained model to each client's specific task, and local differential privacy to prevent reconstruction attacks on the global model. DIPOFLAIR Algorithm is based on a constrained optimization problem that is solved using a Lagrangian approach with DIPOFEL Algorithm as a subroutine. The simulation result shows that the regret bounds of the proposed algorithms are better than the state-of-the-art algorithms.