Coverage estimation using statistics of the extremes for when testing reveals no failures

• Published in: IEEE transactions on computers 2002-01, Vol.51 (1), p.3-12
• Main Authors: Kaufman, L.M., Johnson, B.W., Dugan, J.B.
• Format: Article
• Language: English
• Subjects: Asymptotic properties; Classification; Estimates; Extreme values; Failure; Faults; Mathematical models; Statistical analysis; Statistics; Studies; Testing
• ISSN: 0018-9340; 1557-9956
• DOI: 10.1109/12.980013

The existing classes of fault coverage models require an a priori distribution for collected data in their analysis. Using these models, analyses can be performed using various assumed distributions. The assumed distributions may not accurately reflect the behavior of the collected data and, as a result, the coverage values predicted by the models may be inaccurate, especially if testing yields little or no failure data. Since the occurrence of an uncovered fault in an ultra-dependable system is a rare event, then statistics of the extremes can be used to quantify uncoverage estimates in such systems. Statistics of the extremes provides for an analysis of rare event data without requiring any a priori knowledge of its distribution. It classifies most distributions into one of three asymptotic families; that is, in the limit, most distributions converge to one of three forms. Using statistics of the extremes, a coverage model is developed for when testing reveals no failures. From this model, the number of fault injection experiments required to demonstrate that a desired coverage level can be met is derived, as is the probability that this coverage estimate can be met.