DAOS as HPC Storage, a view from Numerical Weather Prediction

Object storage solutions potentially address long-standing performance issues with POSIX file systems for certain I/O workloads, and new storage technologies offer promising performance characteristics for data-intensive use cases. In this work, we present a preliminary assessment of Intel's Di...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2023-02
Main Authors: Nicolau Manubens, Quintino, Tiago, Smart, Simon D, Danovaro, Emanuele, Jackson, Adrian
Format: Article
Language:English
Subjects:
Benchmarks
Configuration management
Numerical prediction
Numerical weather forecasting
Storage systems
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Object storage solutions potentially address long-standing performance issues with POSIX file systems for certain I/O workloads, and new storage technologies offer promising performance characteristics for data-intensive use cases. In this work, we present a preliminary assessment of Intel's Distributed Asynchronous Object Store (DAOS), an emerging high-performance object store, in conjunction with non-volatile storage and evaluate its potential use for HPC storage. We demonstrate DAOS can provide the required performance, with bandwidth scaling linearly with additional DAOS server nodes in most cases, although choices in configuration and application design can impact achievable bandwidth. We describe a new I/O benchmark and associated metrics that address object storage performance from application-derived workloads.
ISSN:2331-8422