Google’s MapReduce programming model — Revisited

Google’s MapReduce programming model serves for processing large data sets in a massively parallel manner. We deliver the first rigorous description of the model including its advancement as Google’s domain-specific language Sawzall. To this end, we reverse-engineer the seminal papers on MapReduce a...

Full description

Saved in:
Bibliographic Details
Published in:Science of computer programming 2008-01, Vol.70 (1), p.1-30
Main Author: Lämmel, Ralf
Format: Article
Language:English
Subjects:
Data processing
Distributed programming
Executable specification
Haskell
List homomorphism
MapReduce
Parallel programming
Reduce
Sawzall
Software design
Typed functional programming
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Google’s MapReduce programming model serves for processing large data sets in a massively parallel manner. We deliver the first rigorous description of the model including its advancement as Google’s domain-specific language Sawzall. To this end, we reverse-engineer the seminal papers on MapReduce and Sawzall, and we capture our findings as an executable specification. We also identify and resolve some obscurities in the informal presentation given in the seminal papers. We use typed functional programming (specifically Haskell) as a tool for design recovery and executable specification. Our development comprises three components: (i) the basic program skeleton that underlies MapReduce computations; (ii) the opportunities for parallelism in executing MapReduce computations; (iii) the fundamental characteristics of Sawzall’s aggregators as an advancement of the MapReduce approach. Our development does not formalize the more implementational aspects of an actual, distributed execution of MapReduce computations.
ISSN:0167-6423
1872-7964
DOI:10.1016/j.scico.2007.07.001