Generalized Coverage Criteria for Combinatorial Sequence Testing

We present a new model-based approach for testing systems that use sequences of actions and assertions as test vectors. Our solution includes a method for quantifying testing quality, a tool for generating high-quality test suites based on the coverage criteria we propose, and a framework for assess...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on software engineering 2023-08, Vol.49 (8), p.4023-4034
Main Authors: Elyasaf, Achiya, Farchi, Eitan, Margalit, Oded, Weiss, Gera, Weiss, Yeshayahu
Format: Article
Language:English
Subjects:
Automata
Bayes methods
Bayesian analysis
Bayesian risk-Reduction
behavioral programming
Behavioral sciences
Codes
Combinatorial analysis
combinatorial test design
Computer bugs
Criteria
model-based testing
sequence testing
Synchronization
test coverage
test generation
test optimization
Testing
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We present a new model-based approach for testing systems that use sequences of actions and assertions as test vectors. Our solution includes a method for quantifying testing quality, a tool for generating high-quality test suites based on the coverage criteria we propose, and a framework for assessing risks. For testing quality, we propose a method that specifies generalized coverage criteria over sequences of actions, which extends previous approaches. Our publicly available tool demonstrates how to extract effective test suites from test plans based on these criteria. We also present a Bayesian approach for measuring the probabilities of bugs or risks, and show how this quantification can help achieve an informed balance between exploitation and exploration in testing. Finally, we provide an empirical evaluation demonstrating the effectiveness of our tool in finding bugs, assessing risks, and achieving coverage.
ISSN:0098-5589
1939-3520
DOI:10.1109/TSE.2023.3279570