Experimental Design Networks: A Paradigm for Serving Heterogeneous Learners Under Networking Constraints

Significant advances in edge computing capabilities enable learning to occur at geographically diverse locations. In general, the training data needed in those learning tasks are not only heterogeneous but also not fully generated locally. In this paper, we propose an experimental design network par...

Full description

Saved in:
Bibliographic Details
Published in:IEEE/ACM transactions on networking 2023-10, Vol.31 (5), p.1-15
Main Authors: Li, Yuanyuan, Liu, Yuezhou, Su, Lili, Yeh, Edmund, Ioannidis, Stratis
Format: Article
Language:English
Subjects:
Algorithms
Bayes methods
Bayesian linear regression
Cognitive tasks
Continuity (mathematics)
Data models
Data transmission
Design of experiments
DR-submodularity
Edge computing
Experimental design
Learning
Linear regression
Network topology
Nodes
Optimization
Regression models
Task analysis
Training
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Significant advances in edge computing capabilities enable learning to occur at geographically diverse locations. In general, the training data needed in those learning tasks are not only heterogeneous but also not fully generated locally. In this paper, we propose an experimental design network paradigm, wherein learner nodes train possibly different Bayesian linear regression models via consuming data streams generated by data source nodes over a network. We formulate this problem as a social welfare optimization problem in which the global objective is defined as the sum of experimental design objectives of individual learners, and the decision variables are the data transmission strategies subject to network constraints. We first show that, assuming Poisson data streams in steady state, the global objective is a continuous DR-submodular function. We then propose a Frank-Wolfe type algorithm that outputs a solution within a 1-1/e factor from the optimal. Our algorithm contains a novel gradient estimation component which is carefully designed based on Poisson tail bounds and sampling. Finally, we complement our theoretical findings through extensive experiments. Our numerical evaluation shows that the proposed algorithm outperforms several baseline algorithms both in maximizing the global objective and in the quality of the trained models.
ISSN:1063-6692
1558-2566
DOI:10.1109/TNET.2023.3243534