CPU Frequency Tuning to Improve Energy Efficiency of MapReduce Systems

Energy efficiency is a major concern in today's data centers that house large scale distributed processing systems such as data parallel MapReduce clusters. Modern power aware systems utilize the dynamic voltage and frequency scaling mechanism available in processors to manage the energy consum...

Full description

Saved in:
Bibliographic Details
Main Authors: Tiwari, Nidhi, Bellur, Umesh, Sarkar, Santonu, Indrawan, Maria
Format: Conference Proceeding
Language:English
Subjects:
CPU-Frequency Tuning
Data models
Distributed Computing
Distributed databases
Electronic mail
Energy consumption
Energy Efficiency
MapReduce
Predictive models
Program processors
Tuning
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Energy efficiency is a major concern in today's data centers that house large scale distributed processing systems such as data parallel MapReduce clusters. Modern power aware systems utilize the dynamic voltage and frequency scaling mechanism available in processors to manage the energy consumption. In this paper, we initially characterize the energy efficiency of MapReduce jobs with respect to built-in power governors. Our analysis indicates that while a built-in power governor provides the best energy efficiency for a job that is CPU as well as IO intensive, a common CPU-frequency across the cluster provides best the energy efficiency for other types of jobs. In order to identify this optimal frequency setting, we derive energy and performance models for MapReduce jobs on a HPC cluster and validate these models experimentally on different platforms. We demonstrate how these models can be used to improve energy efficiency of the machine learning MapReduce applications running on the Yarn platform. The execution of jobs at their optimal frequencies improves the energy efficiency by average 25% over the default governor setting. In case of mixed workloads, the energy efficiency improves by up to 10% when we use an optimal CPU-frequency across the cluster.
ISSN:1521-9097
2690-5965
DOI:10.1109/ICPADS.2016.0135