Exploiting redundancy to boost performance in a RAID-10 style cluster-based file system

While aggregating the throughput of existing disks on cluster nodes is a cost-effective approach to alleviate the I/O bottleneck in cluster computing, this approach suffers from potential performance degradations due to contentions for shared resources on the same node between storage data processin...

Full description

Saved in:
Bibliographic Details
Published in:Cluster computing 2006-10, Vol.9 (4), p.433-447
Main Authors: Zhu, Yifeng, Jiang, Hong, Qin, Xiao, Feng, Dan, Swanson, David R.
Format: Article
Language:English
Subjects:
Algorithms
Clusters
Data processing
Heuristic scheduling
Nodes
Performance degradation
Performance enhancement
RAID
Redundancy
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:While aggregating the throughput of existing disks on cluster nodes is a cost-effective approach to alleviate the I/O bottleneck in cluster computing, this approach suffers from potential performance degradations due to contentions for shared resources on the same node between storage data processing and user task computation. This paper proposes to judiciously utilize the storage redundancy in the form of mirroring existed in a RAID-10 style file system to alleviate this performance degradation. More specifically, a heuristic scheduling algorithm is developed, motivated from the observations of a simple cluster configuration, to spatially schedule write operations on the nodes with less load among each mirroring pair. The duplication of modified data to the mirroring nodes is performed asynchronously in the background. The read performance is improved by two techniques: doubling the degree of parallelism and hot-spot skipping. A synthetic benchmark is used to evaluate these algorithms in a real cluster environment and the proposed algorithms are shown to be very effective in performance enhancement.
ISSN:1386-7857
1573-7543
DOI:10.1007/s10586-006-0011-6