A federated learning algorithm using parallel-ensemble method on non-IID datasets

• Published in: Complex & intelligent systems 2023-12, Vol.9 (6), p.6891-6903
• Main Authors: Yu, Haoran, Wu, Chang, Yu, Haixin, Wei, Xuelin, Liu, Siyan, Zhang, Ying
• Format: Article
• Language: English
• Subjects: Complexity; Computational Intelligence; Data Structures and Information Theory; Engineering; Ensemble learning; Federated learning; Image classification; Non-IID; Original Article
• ISSN: 2199-4536; 2198-6053
• DOI: 10.1007/s40747-023-01110-7

Traditional federated learning algorithms suffer from considerable performance reduction with non-identically and independently distributed datasets. This paper proposes a federated learning algorithm based on parallel-ensemble learning, which improves performance for image classification on these datasets. The training process of this algorithm includes basic federation learning and meta federation learning. First, several basic models are trained through cross-validation of federated learning, and then the meta-model is trained using the prediction results of the validation sets. In the training process, the training of different basic models is parallel. In prediction, meta-model is used to aggregate the output of the basic models to get the final prediction results. Our algorithm can achieve higher accuracy than traditional federated learning when using non-independent identically distributed datasets for image classification. Our algorithm aggregates different models through federated learning based on parallel-ensemble method, and improves the image classification performance of federated learning on non-independent identically distributed datasets.