A Peer-to-Peer Architecture for Supporting Dynamic Shared Libraries in Large-Scale Systems

Historically, scientific computing applications have been statically linked before running on massively parallel High Performance Computing (HPC) platforms. In recent years, demand for supporting dynamically linked applications at large scale has increased. When programs running at large scale dynam...

Full description

Saved in:
Bibliographic Details
Main Authors: Dosanjh, M. G. F., Bridges, P. G., Kelly, S. M., Laros, J. H.
Format: Conference Proceeding
Language:English
Subjects:
DSL
File systems
Fuses
High performance computing
Libraries
Optimization
Parallel programming
Peer to peer computing
Protocols
Runtime
System software
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Historically, scientific computing applications have been statically linked before running on massively parallel High Performance Computing (HPC) platforms. In recent years, demand for supporting dynamically linked applications at large scale has increased. When programs running at large scale dynamically load shared objects, they often request the same file from shared storage. These independent requests tax the shared storage and the network, causing a significant delay in computation time. In this paper, we propose to leverage a proven file sharing technique, Bit Torrent, abstracted by an on-node FUSE interface to create a system-level distribution method for these files. We detail our proposed methodology, related work, and our current progress.
ISSN:0190-3918
2332-5690
DOI:10.1109/ICPPW.2012.11