Revisiting, Benchmarking and Exploring API Recommendation: How Far Are We?

Application Programming Interfaces (APIs), which encapsulate the implementation of specific functions as interfaces, greatly improve the efficiency of modern software development. As the number of APIs grows up fast nowadays, developers can hardly be familiar with all the APIs and usually need to se...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on software engineering 2023-04, Vol.49 (4), p.1876-1897
Main Authors: Peng, Yun, Li, Shuqing, Gu, Wenwei, Li, Yichen, Wang, Wenxuan, Gao, Cuiyun, Lyu, Michael R.
Format: Article
Language:English
Subjects:
API recommendation
Application programming interface
benchmark
Benchmark testing
Benchmarks
Codes
empirical study
Java
Knowledge based systems
Queries
Recommender systems
Soft sensors
Software development
Task analysis
Tutorials
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Application Programming Interfaces (APIs), which encapsulate the implementation of specific functions as interfaces, greatly improve the efficiency of modern software development. As the number of APIs grows up fast nowadays, developers can hardly be familiar with all the APIs and usually need to search for appropriate APIs for usage. So lots of efforts have been devoted to improving the API recommendation task. However, it has been increasingly difficult to gauge the performance of new models due to the lack of a uniform definition of the task and a standardized benchmark. For example, some studies regard the task as a code completion problem, while others recommend relative APIs given natural language queries. To reduce the challenges and better facilitate future research, in this paper, we revisit the API recommendation task and aim at benchmarking the approaches. Specifically, the paper groups the approaches into two categories according to the task definition, i.e., query-based API recommendation and code-based API recommendation. We study 11 recently-proposed approaches along with 4 widely-used IDEs. One benchmark named APIBench is then built for the two respective categories of approaches. Based on APIBench, we distill some actionable insights and challenges for API recommendation. We also achieve some implications and directions for improving the performance of recommending APIs, including appropriate query reformulation, data source selection, low resource setting, user-defined APIs, and query-based API recommendation with usage patterns.
ISSN:0098-5589
1939-3520
DOI:10.1109/TSE.2022.3197063