Imperative or Functional Control Flow Handling: Why not the Best of Both Worlds?

Modern data analysis tasks often involve control flow statements, such as the iterations in PageRank and K-means. To achieve scalability, developers usually implement these tasks in distributed dataflow systems, such as Spark and Flink. Designers of such systems have to choose between providing impe...

Full description

Saved in:
Bibliographic Details
Published in:SIGMOD record 2022-05, Vol.51 (1), p.60-67
Main Authors: Gévay, Gábor E., Rabl, Tilmann, Breß, Sebastian, Madai-Tahy, Loránd, Quiané-Ruiz, Jorge-Arnulfo, Markl, Volker
Format: Article
Language:English
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Modern data analysis tasks often involve control flow statements, such as the iterations in PageRank and K-means. To achieve scalability, developers usually implement these tasks in distributed dataflow systems, such as Spark and Flink. Designers of such systems have to choose between providing imperative or functional control flow constructs to users. Imperative constructs are easier to use, but functional constructs are easier to compile to an efficient dataflow job. We propose Mitos, a system where control flow is both easy to use and efficient. Mitos relies on an intermediate representation based on the static single assignment form. This allows us to abstract away from specific control flow constructs and treat any imperative control flow uniformly both when building the dataflow job and when coordinating the distributed execution.
ISSN:0163-5808
DOI:10.1145/3542700.3542715