DynaComm: Accelerating Distributed CNN Training Between Edges and Clouds Through Dynamic Communication Scheduling

To reduce uploading bandwidth and address privacy concerns, deep learning at the network edge has been an emerging topic. Typically, edge devices collaboratively train a shared model using real-time generated data through the Parameter Server framework. Although all the edge devices can share the co...

Full description

Saved in:
Bibliographic Details
Published in:IEEE journal on selected areas in communications 2022-02, Vol.40 (2), p.611-625
Main Authors: Cai, Shangming, Wang, Dongsheng, Wang, Haixia, Lyu, Yongqiang, Xu, Guangquan, Zheng, Xi, Vasilakos, Athanasios V.
Format: Article
Language:English
Subjects:
Artificial neural networks
Computational modeling
convolutional neural network
Deep learning
deep learning training
Distribuerade datorsystem
Dynamic scheduling
Edge computing
Mathematical models
Model accuracy
Parameters
Performance evaluation
Pervasive Mobile Computing
Processor scheduling
Scheduling
Servers
Training
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:To reduce uploading bandwidth and address privacy concerns, deep learning at the network edge has been an emerging topic. Typically, edge devices collaboratively train a shared model using real-time generated data through the Parameter Server framework. Although all the edge devices can share the computing workloads, the distributed training processes over edge networks are still time-consuming due to the parameters and gradients transmission procedures between parameter servers and edge devices. Focusing on accelerating distributed Convolutional Neural Networks (CNNs) training at the network edge, we present DynaComm, a novel scheduler that dynamically decomposes each transmission procedure into several segments to achieve optimal layer-wise communications and computations overlapping during run-time. Through experiments, we verify that DynaComm manages to achieve optimal layer-wise scheduling for all cases compared to competing strategies while the model accuracy remains untouched.
ISSN:0733-8716
1558-0008
1558-0008
DOI:10.1109/JSAC.2021.3118419