PARS-Push: Personalized, Asynchronous and Robust Decentralized Optimization

We study the multi-step Model-Agnostic Meta-Learning (MAML) framework where a group of n agents seeks to find a common point that enables "few-shot" learning (personalization) via local stochastic gradient steps on their local functions. We formulate the personalized optimization problem...

Full description

Saved in:
Bibliographic Details
Published in:IEEE control systems letters 2023, Vol.7, p.361-366
Main Authors: Toghani, Mohammad Taha, Lee, Soomin, Uribe, Cesar A.
Format: Article
Language:English
Subjects:
Asynchronous communication
asynchronous communications
Convergence
Costs
Decentralized optimization
Delays
few-shot learning
model-agnostic meta-learning
Optimization
personalization
Stochastic processes
Task analysis
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We study the multi-step Model-Agnostic Meta-Learning (MAML) framework where a group of n agents seeks to find a common point that enables "few-shot" learning (personalization) via local stochastic gradient steps on their local functions. We formulate the personalized optimization problem under the MAML framework and propose PARS-Push, a decentralized asynchronous algorithm robust to message failures, communication delays, and directed message sharing. We characterize the convergence rate of PARS-Push under arbitrary multi-step personalization for smooth strongly convex, and smooth non-convex functions. Moreover, we provide numerical experiments showing its performance under heterogeneous data setups.
ISSN:2475-1456
2475-1456
DOI:10.1109/LCSYS.2022.3189317