Adaptive prefetching and storage reorganization in a log-structured storage system

We present a storage management system that has the ability to adapt to the data access characteristics of the application that uses it based on collection and analysis of runtime statistics. This feature is especially useful in the storage management layer of database systems, where applications ex...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on knowledge and data engineering 1998-09, Vol.10 (5), p.824-838
Main Authors: Chye Lin Chee, Hongjun Lu, Hong Tang, Ramamoorthy, C.V.
Format: Article
Language:English
Subjects:
Application software
Applied sciences
Computer science
Computer science
control theory
systems
Computer Society
Database systems
Delay effects
Exact sciences and technology
Information systems. Data bases
Memory organisation. Data processing
Object oriented modeling
Prefetching
Prototypes
Software
Software engineering
Statistical analysis
Statistics
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We present a storage management system that has the ability to adapt to the data access characteristics of the application that uses it based on collection and analysis of runtime statistics. This feature is especially useful in the storage management layer of database systems, where applications exhibit relatively predictable access patterns. Adaptive reorganization is performed by the storage management system in a manner that optimizes the access patterns of the system for which it is used. We enhance the log-structured storage system that naturally caters for write optimization, with the addition of a statistics collection mechanism to determine data access patterns of applications. The storage system can serve as a testbed for a variety of statistics analysis and clustering mechanisms. Higher level application-specific data clustering mechanisms can be used to override the storage system's low-level clustering mechanisms. In addition, the analysis techniques and reorganization scheme can be used in other storage systems. Performance results from our prototype show potential response time speedups of up to 83 percent over the basic log-structured file system in the best case, using a combination of storage reorganization and prefetching.
ISSN:1041-4347
1558-2191
DOI:10.1109/69.729739