Contrastive and Non-Contrastive Strategies for Federated Self-Supervised Representation Learning and Deep Clustering

We investigate federated self-supervised representation learning (FedSSRL) and federated clustering (FedCl), aiming to derive low-dimensional representations of datasets distributed across multiple clients, potentially in a heterogeneous manner. Our proposed solutions for both FedSSRL and FedCl invo...

Full description

Saved in:
Bibliographic Details
Published in:IEEE journal of selected topics in signal processing 2024-09, p.1-16
Main Authors: Miao, Runxuan, Koyuncu, Erdem
Format: Article
Language:English
Subjects:
clustering
Contrastive learning
Data models
Distributed databases
Federated learning
Representation learning
Signal processing algorithms
Training
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We investigate federated self-supervised representation learning (FedSSRL) and federated clustering (FedCl), aiming to derive low-dimensional representations of datasets distributed across multiple clients, potentially in a heterogeneous manner. Our proposed solutions for both FedSSRL and FedCl involves a comparative analysis from a broad learning context. In particular, we show that a two-stage model, beginning with representation learning and followed by clustering, is an effective learning strategy for both tasks. Notably, integrating a contrastive loss as regularizer significantly boosts performance, even if the task is representation learning. Moreover, for FedCl, a contrastive loss is most effective in both stages, whereas FedSSRL benefits more from a non-contrastive loss. These findings are corroborated by extensive experiments on various image datasets
ISSN:1932-4553
1941-0484
DOI:10.1109/JSTSP.2024.3461311