HPC Data Storage at a Glance: The Santos Dumont Experience

High-Performance Computing (HPC) platforms are used to solve the most diverse scientific problems in research areas, such as biology, chemistry, physics, and health sciences. Researchers use a multitude of scientific software, which have different requirements. These requirements include input and o...

Full description

Saved in:
Bibliographic Details
Main Authors: Carneiro, Andre Ramos, Bez, Jean Luca, Osthoff, Carla, Schnorr, Lucas Mello, Navaux, Philippe O. A.
Format: Conference Proceeding
Language:English
Subjects:
File systems
High-Performance Storage
I/O Characterization
I/O Workload
Lustre
Memory
Metadata
Parallel File System
Scalability
Supercomputers
System performance
Usability
Visualization
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:High-Performance Computing (HPC) platforms are used to solve the most diverse scientific problems in research areas, such as biology, chemistry, physics, and health sciences. Researchers use a multitude of scientific software, which have different requirements. These requirements include input and output operations, which directly impact performance due to the existing difference in processing and data access speeds. Thus, supercomputers must efficiently handle a mixed workload scenario when storing data from the applications. Knowledge of the application set and its performance running in a supercomputer is needed to understand the storage system's usage, pinpoint possible bottlenecks, and guide optimization techniques. This research proposes a methodology and visualization tool to evaluate a supercomputer's data storage infrastructure's performance, taking into account the diverse workload and demands of the system over a long period of operation. As a study case, we focus on the Santos Dumont supercomputer, where we were able to identify inefficient usage and problematic factors of performance.
ISSN:2643-3001
DOI:10.1109/SBAC-PAD53543.2021.00027