Fault domain-based testing in imperfect situations: a heuristic approach and case studies

• Published in: Software quality journal 2015-09, Vol.23 (3), p.423-452
• Main Authors: Belli, Fevzi, Beyazıt, Mutlu, Endo, Andre Takeshi, Mathur, Aditya, Simao, Adenilso
• Format: Article
• Language: English
• Subjects: Case studies; Compilers; Computer programs; Computer Science; Data Structures and Information Theory; Empirical analysis; Experiments; Faults; Finite state machines; Heuristic; Heuristic methods; Interpreters; Methods; Mutations; Operating Systems; Programming Languages; Quality control; Software; Software engineering; Software Engineering/Programming and Operating Systems; Studies; Test sets; Testing
• ISSN: 0963-9314; 1573-1367
• DOI: 10.1007/s11219-014-9242-6

Model-based testing (MBT) involves creating an abstraction, called a model, to represent the system and automatically deriving test cases from this model. MBT can be performed using various approaches that generally employ certain assumptions or requirements affecting the test performance in practice. Here, we consider the harmonized state identifiers (HSI) method, which is based on finite state machine (FSM) models and generates test sets that cover all faults in a given domain under certain conditions. We are interested in the application of the HSI method in practical scenarios where some conditions do not hold or are not straightforward to satisfy. Thus, we propose a heuristic extension to the HSI method, called heuristic HSI (HHSI), to consider imperfect situations as they often occur in practice. To analyze the characteristics of HHSI, we empirically compare it to random testing and coverage-based testing using non-trivial case studies. The experiments include model-based mutation analyses over several FSM models.