AR-Dedupe： An Efficient Deduplication Approach for Cluster Deduplication System

As data are growing rapidly in data centers, inline cluster deduplication technique has been widely used to improve storage efficiency and data reliability. However, there are some challenges faced by the cluster deduplication system： the decreasing data deduplication rate with the increasing dedupl...

Full description

Saved in:
Bibliographic Details
Published in:Shanghai jiao tong da xue xue bao 2015-02, Vol.20 (1), p.76-81
Main Author: 邢平轩 肖侬 刘芳 孙振 何晚辉
Format: Article
Language:English
Subjects:
Algorithms
Architecture
Clusters
Communication systems
Computer Science
Data centers
Electrical Engineering
Engineering
Life Sciences
Load balancing
Materials Science
Performance enhancement
Performance indices
Servers
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:As data are growing rapidly in data centers, inline cluster deduplication technique has been widely used to improve storage efficiency and data reliability. However, there are some challenges faced by the cluster deduplication system： the decreasing data deduplication rate with the increasing deduplication server nodes, high communication overhead for data routing, and load balance to improve the throughput of the system. In this paper, we propose a well-performed cluster deduplication system called AR-Dedupe. The experimental results of two real datasets demonstrate that AR-Dedupe can achieve a high data deduplication rate with a low communication overhead and keep the system load balancing well at the same time through a new data routing algorithm. In addition, we utilize application-aware mechanism to speed up the index of handprints in the routing server which has a 30% performance improvement.
ISSN:1007-1172
1995-8188
DOI:10.1007/s12204-015-1591-1