The Relation of Test-Related Factors to Software Quality: A Case Study on Apache Systems

• Published in: Empirical software engineering : an international journal 2021-03, Vol.26 (2), Article 18
• Main Authors: Pecorelli, Fabiano, Palomba, Fabio, De Lucia, Andrea
• Format: Article
• Language: English
• Subjects: Case studies; Compilers; Computer Science; Defects; Interpreters; Programming Languages; Software; Software Engineering/Programming and Operating Systems; Software quality
• ISSN: 1382-3256; 1573-7616
• DOI: 10.1007/s10664-020-09891-y

Testing represents a crucial activity to ensure software quality. Recent studies have shown that test-related factors (e.g., code coverage) can be reliable predictors of software code quality, as measured by post-release defects. While these studies provided initial compelling evidence on the relation between tests and post-release defects, they considered different test-related factors separately: as a consequence, there is still a lack of knowledge of whether these factors are still good predictors when considering all together. In this paper, we propose a comprehensive case study on how test-related factors relate to production code quality in Apache systems. We first investigated how the presence of tests relates to post-release defects; then, we analyzed the role played by the test-related factors previously shown as significantly related to post-release defects. The key findings of the study show that, when controlling for other metrics (e.g., size of the production class), test-related factors have a limited connection to post-release defects.