Cost Minimization for Big Data Processing in Geo-Distributed Data Centers

The explosive growth of demands on big data processing imposes a heavy burden on computation, storage, and communication in data centers, which hence incurs considerable operational expenditure to data center providers. Therefore, cost minimization has become an emergent issue for the upcoming big d...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on emerging topics in computing 2014-09, Vol.2 (3), p.314-323
Main Authors: Gu, Lin, Zeng, Deze, Li, Peng, Guo, Song
Format: Article
Language:English
Subjects:
Big data
Data handling
Data storage systems
Distributed databases
Information management
Minimization
Routing protocols
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The explosive growth of demands on big data processing imposes a heavy burden on computation, storage, and communication in data centers, which hence incurs considerable operational expenditure to data center providers. Therefore, cost minimization has become an emergent issue for the upcoming big data era. Different from conventional cloud services, one of the main features of big data services is the tight coupling between data and computation as computation tasks can be conducted only when the corresponding data are available. As a result, three factors, i.e., task assignment, data placement, and data movement, deeply influence the operational expenditure of data centers. In this paper, we are motivated to study the cost minimization problem via a joint optimization of these three factors for big data services in geo-distributed data centers. To describe the task completion time with the consideration of both data transmission and computation, we propose a 2-D Markov chain and derive the average task completion time in closed-form. Furthermore, we model the problem as a mixed-integer nonlinear programming and propose an efficient solution to linearize it. The high efficiency of our proposal is validated by extensive simulation-based studies.
ISSN:2168-6750
2168-6750
DOI:10.1109/TETC.2014.2310456