Automated Classification of Overfitting Patches With Statically Extracted Code Features

Automatic program repair (APR) aims to reduce the cost of manually fixing software defects. However, APR suffers from generating a multitude of overfitting patches, those patches that fail to correctly repair the defect beyond making the tests pass. This paper presents a novel overfitting patch dete...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on software engineering 2022-08, Vol.48 (8), p.2920-2938
Main Authors: Ye, He, Gu, Jian, Martinez, Matias, Durieux, Thomas, Monperrus, Martin
Format: Article
Language:English
Subjects:
Automatic program repair
Classification
Code features
Defects
Evaluation
Feature extraction
Java
Maintenance engineering
Overfitting patch
Patch assessment
Patches (structures)
Predictive models
Probabilistic models
Repair
Software
Syntactics
Training
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Automatic program repair (APR) aims to reduce the cost of manually fixing software defects. However, APR suffers from generating a multitude of overfitting patches, those patches that fail to correctly repair the defect beyond making the tests pass. This paper presents a novel overfitting patch detection system called ODS to assess the correctness of APR patches. ODS first statically compares a patched program and a buggy program in order to extract code features at the abstract syntax tree (AST) level, for the single programming language Java. Then, ODS uses supervised learning with the captured code features and patch correctness labels to automatically learn a probabilistic model. The learned ODS model can then finally be applied to classify new and unseen program repair patches. We conduct a large-scale experiment to evaluate the effectiveness of ODS on patch correctness classification based on 10,302 patches from Defects4J, Bugs.jar and Bears benchmarks. The empirical evaluation shows that ODS is able to correctly classify 71.9 percent of program repair patches from 26 projects, which improves the state-of-the-art. ODS is applicable in practice and can be employed as a post-processing procedure to classify the patches generated by different APR systems.
ISSN:0098-5589
1939-3520
1939-3520
DOI:10.1109/TSE.2021.3071750