Interpreting a Successful Testing Process: Risk and Actual Coverage

Testing is inherently incomplete; no test suite will ever be able to test all possible usage scenarios of a system. It is therefore vital to assess the implication of a system passing a test suite. This paper quantifies that implication by means of two distinct, but related, measures: the risk quant...

Full description

Saved in:
Bibliographic Details
Main Authors: Stoelinga, M., Timmer, M.
Format: Conference Proceeding
Language:English
Subjects:
Computer science
Costs
coverage
Error probability
formal testing
Mathematical model
probabilistic
Random variables
risk
Risk management
Software engineering
Software testing
Solid modeling
System testing
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Testing is inherently incomplete; no test suite will ever be able to test all possible usage scenarios of a system. It is therefore vital to assess the implication of a system passing a test suite. This paper quantifies that implication by means of two distinct, but related, measures: the risk quantifies the confidence in a system after it passes a test suite, i.e., the number of faults still expected to be present (weighted by their severity); the actual coverage quantifies the extent to which faults have been shown absent, i.e., the fraction of possible faults that has been covered. We provide evaluation algorithms that calculate these metrics for a given test suite, as well as optimisation algorithms that yield the best test suite for a given optimisation criterion.
DOI:10.1109/TASE.2009.26