Exploring the Potential of Pre-Trained Language Models of Code for Automated Program Repair

In the realm of software development, automated program repair (APR) emerges as a pivotal technique, autonomously debugging faulty code to boost productivity. Despite the notable advancements of large pre-trained language models of code (PLMCs) in code generation, their efficacy in complex tasks lik...

Full description

Saved in:
Bibliographic Details
Published in:Electronics (Basel) 2024-04, Vol.13 (7), p.1200
Main Authors: Hao, Sichong, Shi, Xianjun, Liu, Hongwei
Format: Article
Language:English
Subjects:
Adaptability
Architecture
Artificial intelligence
Automation
Computational linguistics
Curricula
Datasets
Debugging
Efficiency
Innovations
Language processing
Learning
Mechanization
Natural language interfaces
Program errors
Programming languages
Repair
Semantics
Software
Software development
Source code
Success
Task complexity
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In the realm of software development, automated program repair (APR) emerges as a pivotal technique, autonomously debugging faulty code to boost productivity. Despite the notable advancements of large pre-trained language models of code (PLMCs) in code generation, their efficacy in complex tasks like APR remains suboptimal. This limitation is attributed to the generic development of PLMCs, whose specialized potential for APR is yet be to fully explored. In this paper, we propose a novel approach designed to enhance PLMCs’ APR performance through source code augmentation and curriculum learning. Our approach employs code augmentation operators to generate a spectrum of syntactically varied yet semantically congruent bug-fixing programs, thus enriching the dataset’s diversity. Furthermore, we design a curriculum learning strategy, enabling PLMCs to develop a deep understanding of program semantics from these enriched code variants, thereby refining their APR fine-tuning prowess. We apply our approach across different PLMCs and systematically evaluate it on three benchmarks: BFP-small, BFP-medium, and Defects4J. The experimental results show that our approach outperforms both original models and existing baseline methods, demonstrating the promising future of adapting PLMCs for code debugging in practice.
ISSN:2079-9292
2079-9292
DOI:10.3390/electronics13071200