A comprehensive empirical investigation on failure clustering in parallel debugging

The clustering technique has attracted a lot of attention as a promising strategy for parallel debugging in multi-fault scenarios, this heuristic approach (i.e., failure indexing or fault isolation) enables developers to perform multiple debugging tasks simultaneously through dividing failed test ca...

Full description

Saved in:
Bibliographic Details
Published in:The Journal of systems and software 2022-11, Vol.193, p.111452, Article 111452
Main Authors: Song, Yi, Xie, Xiaoyuan, Liu, Quanming, Zhang, Xihao, Wu, Xi
Format: Article
Language:English
Subjects:
Failure clustering
Fault isolation
Multiple-fault
Parallel debugging
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The clustering technique has attracted a lot of attention as a promising strategy for parallel debugging in multi-fault scenarios, this heuristic approach (i.e., failure indexing or fault isolation) enables developers to perform multiple debugging tasks simultaneously through dividing failed test cases into several disjoint groups. When using statement ranking representation to model failures for better clustering, several factors influence clustering effectiveness, including the risk evaluation formula (REF), the number of faults (NOF), the fault type (FT), and the number of successful test cases paired with one individual failed test case (NSP1F). In this paper, we present the first comprehensive empirical study of how these four factors influence clustering effectiveness. We conduct extensive controlled experiments on 1060 faulty versions of 228 simulated faults and 141 real faults, and the results reveal that: (1) GP19 is highly competitive across all REFs, (2) clustering effectiveness decreases as NOF increases, (3) higher clustering effectiveness is easier to achieve when a program contains only predicate faults, and (4) clustering effectiveness remains when the scale of NSP1F is reduced to 20%. •The first empirical study regarding the clustering in parallel debugging.•A promising SBFL formula for failure representation in parallel debugging is found.•Clustering effectiveness decreases as the number of faults increases.•Better clustering is easier to obtain when a program has only predicate faults.•Clustering effectiveness remains when successful test cases are reduced to 20%.
ISSN:0164-1212
1873-1228
DOI:10.1016/j.jss.2022.111452