Handling Iterations in Distributed Dataflow Systems

Over the past decade, distributed dataflow systems (DDS) have become a standard technology. In these systems, users write programs in restricted dataflow programming models, such as MapReduce, which enable them to scale out program execution to a shared-nothing cluster of machines. Yet, there is no...

Full description

Saved in:
Bibliographic Details
Published in:ACM computing surveys 2022-12, Vol.54 (9), p.1-38
Main Authors: Gévay, Gábor E., Soto, Juan, Markl, Volker
Format: Article
Language:English
Subjects:
Computer science
Iterative algorithms
Iterative methods
Literature reviews
Programming
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Over the past decade, distributed dataflow systems (DDS) have become a standard technology. In these systems, users write programs in restricted dataflow programming models, such as MapReduce, which enable them to scale out program execution to a shared-nothing cluster of machines. Yet, there is no established consensus that prescribes how to extend these programming models to support iterative algorithms. In this survey, we review the research literature and identify how DDS handle control flow, such as iteration, from both the programming model and execution level perspectives. This survey will be of interest for both users and designers of DDS.
ISSN:0360-0300
1557-7341
DOI:10.1145/3477602