Development of a benchmark to measure system robustness: experiences and lessons learned

Performance benchmarks are used to help decide the question: 'which system is faster?' With the increased use of computers in critical systems, there are more and more applications of resources to improve system quality. However, there are no benchmarks that can be used to compare the depe...

Full description

Saved in:
Bibliographic Details
Main Authors: Byung-Hoon Suh, Hudak, J., Siewiorek, D., Segall, Z.
Format: Conference Proceeding
Language:English
Subjects:
Application software
Computer applications
Computer errors
Hardware
Libraries
Memory management
Programming environments
Robustness
Software performance
Software systems
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Performance benchmarks are used to help decide the question: 'which system is faster?' With the increased use of computers in critical systems, there are more and more applications of resources to improve system quality. However, there are no benchmarks that can be used to compare the dependability and robustness of systems in order to answer the question: 'which system is more reliable?' The authors present an attempt at the development of a benchmark to gauge a system's robustness as measured by its ability to tolerate errors. The initial effort produced four primitive benchmark programs. They include file management system, memory access, user application, and C library functions. Each primitive benchmark targets a system functionality and measures its behavior, given erroneous inputs. The authors present the motivation and experimental results for one of these primitive benchmarks in detail followed by an analysis of the results. A methodology is presented to combine the primitive benchmarks to form an overall robustness figure.< >
DOI:10.1109/ISSRE.1992.285888