The role of bug report evolution in reliable fixing estimation

• Published in: Empirical software engineering : an international journal 2022-12, Vol.27 (7), Article 164
• Main Authors: Vieira, Renan G., Mattos, César Lincoln C., Rocha, Lincoln S., Gomes, João Paulo P., Paixão, Matheus
• Format: Article
• Language: English
• Subjects: Compilers; Computer Science; Configurations; Datasets; Debugging; Evolution; Fixing; Interpreters; Machine learning; Machine Learning Techniques for Software Quality Evaluation (MaLTeSQuE); Open source software; Programming Languages; Software Engineering/Programming and Operating Systems; Software quality; Software upgrading; Task scheduling; Tracking systems
• ISSN: 1382-3256; 1573-7616
• DOI: 10.1007/s10664-022-10213-7

Context Bug reports contain information that can be used by researchers and practitioners to better understand the bug fixing process and to enable the estimation of the effort necessary to fix bugs. In general, estimation models are built using the data (e.g., fixing time, severity, number of comments, number of attachments, and number of patches) present in the reports of fixed bugs (i.e., the report final’s state). However, we claim that this approach is not reliable in a real setting. Effort estimation is necessary for bug fix scheduling and team allocation tasks, which happens closer to the bug report opening than its closing. At that moment, the data available in the bug report is less informative than the data used to build the model, which may lead to an unrealistic estimation. Objective We propose a new approach to estimate bug-fixing time, i.e., the time span between the moment the bug was first reported until the bug is considered fixed. We consider not only the final state of the bug report to create our estimation model but all the previous available states, different from some previous studies that do not consider the reports’ updates. The concept of bug report evolution is used to create a dataset containing all investigated report states. Method First, we verify how often the bug reports and their fields are updated. Next, we evaluate our approach using different machine learning methods as a classification problem, with distinct output configurations, and class balancing techniques. The experimental analysis is performed with data from the JIRA issue tracking system of ten open-source projects. By leveraging the best models (considering all possible configurations) for the different states of the evolution of a bug report, we can assess whether there are significant differences in the models’ estimation ability due to the report’s state. Results We gathered evidence that the reports’ fields are updated often, which characterizes the reports’ evolution, impacting the building of bug-fixing estimation models. The models’ evaluation shows promising results 0.44 up to 0.85, precision values from 0.34 up to 0.74 and recall values from 0.62 up to 0.99, depending on the project. Conclusions Our experiments show that field updates have a meaningful impact on the models’ performance. Furthermore, we present a new approach to deal with the bug report evolution by considering each report version as an independent report. Finally, we also make availab