Automatic Static Vulnerability Detection for Machine Learning Libraries: Are We There Yet?

Automatic detection of software security vulnerabilities is critical in software quality assurance. Many static analysis tools that can help detect security vulnerabilities have been proposed. While these static analysis tools are mainly evaluated on general software projects call into question thei...

Full description

Saved in:
Bibliographic Details
Main Authors: Harzevili, Nima Shiri, Shin, Jiho, Wang, Junjie, Wang, Song, Nagappan, Nachiappan
Format: Conference Proceeding
Language:English
Subjects:
Libraries
Machine learning
machine learning libraries
Social networking (online)
Software
Software quality
Software reliability
Software vulnerabilities
Static analysis
static detection
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Automatic detection of software security vulnerabilities is critical in software quality assurance. Many static analysis tools that can help detect security vulnerabilities have been proposed. While these static analysis tools are mainly evaluated on general software projects call into question their practical effectiveness and usefulness for Machine Learning (ML) libraries. In this paper, we address this question by analyzing five popular and widely used static analysis tools, i.e., Flawfinder, RATS, Cppcheck, Facebook Infer, and Clang static analyzer, on a curated dataset of software security vulnerabilities gathered from four popular ML libraries, including Mlpack, MXNet, PyTorch, and TensorFlow, with a total of 410 known vulnerabilities. Our research categorizes these tools' capabilities to understand better the strengths and weaknesses of the tools for detecting software security vulnerabilities in ML libraries. Overall, our study shows that static analysis tools find a negligible amount of all security vulnerabilities accounting for 5/410 unique vulnerabilities (0.01%), Flawfinder and RATS are the most effective static checkers for finding software security vulnerabilities in ML libraries. We further identify and discuss opportunities to make the tools more effective and practical based on our observations.
ISSN:2332-6549
DOI:10.1109/ISSRE59848.2023.00042