Dynamic Metadata Management for Petabyte-Scale File Systems

In petabyte-scale distributed file systems that decouple read and write from metadata operations, behavior of the metadata server cluster will be critical to overall system performance and scalability. We present a dynamic subtree partitioning and adaptive metadata management system designed to effi...

Full description

Saved in:
Bibliographic Details
Main Authors: Weil, Sage A., Pollack, Kristal T., Brandt, Scott A., Miller, Ethan L.
Format: Conference Proceeding
Language:English
Subjects:
Adaptive systems
Computer architecture
Context modeling
File servers
File systems
Permission
Scalability
Scientific computing
Secure storage
System performance
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In petabyte-scale distributed file systems that decouple read and write from metadata operations, behavior of the metadata server cluster will be critical to overall system performance and scalability. We present a dynamic subtree partitioning and adaptive metadata management system designed to efficiently manage hierarchical metadata workloads that evolve over time. We examine the relative merits of our approach in the context of traditional workload partitioning strategies, and demonstrate the performance, scalability and adaptability advantages in a simulation environment.
DOI:10.1109/SC.2004.22