Overcoming Challenges to Continuous Integration in HPC

Continuous integration (CI) has become a ubiquitous practice in modern software development, with major code hosting services offering free automation on popular platforms. CI offers major benefits, as it enables detecting bugs in code prior to committing changes. While high-performance computing (H...

Full description

Saved in:
Bibliographic Details
Published in:Computing in science & engineering 2022-11, Vol.24 (6), p.54-59
Main Authors: Gamblin, Todd, Katz, Daniel S., Carver, Jeffrey, Morris, Karla
Format: Article
Language:English
Subjects:
Authentication
Automation
Codes
Computer bugs
High performance computing
Maintenance engineering
MATHEMATICS AND COMPUTING
Platforms
Software
Software development
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Continuous integration (CI) has become a ubiquitous practice in modern software development, with major code hosting services offering free automation on popular platforms. CI offers major benefits, as it enables detecting bugs in code prior to committing changes. While high-performance computing (HPC) research relies heavily on software, HPC machines are not considered “common” platforms. This presents several challenges that hinder the adoption of CI in HPC environments, making it difficult to maintain bug-free HPC projects, and resulting in adverse effects on the research community. In this article, we explore the challenges that impede HPC CI, such as hardware diversity, security, isolation, administrative policies, and nonstandard authentication, environments, and job submission mechanisms. We propose several solutions that could enhance the quality of HPC software and the experience of developers. Implementing these solutions would require significant changes at HPC centers, but if these changes are made, it would ultimately enable faster and better science.
ISSN:1521-9615
1558-366X
DOI:10.1109/MCSE.2023.3263458