Evaluating MapReduce for Multi-core and Multiprocessor Systems

This paper evaluates the suitability of the MapReduce model for multi-core and multi-processor systems. MapReduce was created by Google for application development on data-centers with thousands of servers. It allows programmers to write functional-style code that is automatically parallelized and s...

Full description

Saved in:
Bibliographic Details
Main Authors: Ranger, C., Raghuraman, R., Penmetsa, A., Bradski, G., Kozyrakis, C.
Format: Conference Proceeding
Language:English
Subjects:
Concurrent computing
Dynamic scheduling
Fault tolerance
Laboratories
Multiprocessing systems
Parallel programming
Processor scheduling
Programming profession
Runtime
Yarn
Citations: Items that cite this one
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper evaluates the suitability of the MapReduce model for multi-core and multi-processor systems. MapReduce was created by Google for application development on data-centers with thousands of servers. It allows programmers to write functional-style code that is automatically parallelized and scheduled in a distributed system. We describe Phoenix, an implementation of MapReduce for shared-memory systems that includes a programming API and an efficient runtime system. The Phoenix runtime automatically manages thread creation, dynamic task scheduling, data partitioning, and fault tolerance across processor nodes. We study Phoenix with multi-core and symmetric multiprocessor systems and evaluate its performance potential and error recovery features. We also compare MapReduce code to code written in lower-level APIs such as P-threads. Overall, we establish that, given a careful implementation, MapReduce is a promising model for scalable performance on shared-memory systems with simple parallel code
ISSN:1530-0897
2378-203X
DOI:10.1109/HPCA.2007.346181