Exploring Object Stores for High-Energy Physics Data Storage

• Published in: EPJ Web of conferences 2021, Vol.251, p.2066
• Main Authors: López-Gómez, Javier, Blomer, Jakob
• Format: Article
• Language: English
• Subjects: Data retrieval; Data storage; Energy storage; Hardware; High energy physics; Network latency; Performance evaluation; Storage systems
• ISSN: 2100-014X; 2101-6275; 2100-014X
• DOI: 10.1051/epjconf/202125102066

Over the last two decades, ROOT TTree has been used for storing over one exabyte of High-Energy Physics (HEP) events. The TTree columnar on-disk layout has been proved to be ideal for analyses of HEP data that typically require access to many events, but only a subset of the information stored for each of them. Future colliders, and particularly HL-LHC, will bring an increase of at least one order of magnitude in the volume of generated data. Therefore, the use of modern storage hardware, such as low-latency high-bandwidth NVMe devices and distributed object stores, becomes more important. However, TTree was not designed to optimally exploit modern hardware and may become a bottleneck for data retrieval. The ROOT RNTuple I/O system aims at overcoming TTree’s limitations and at providing improved effciency for modern storage systems. In this paper, we extend RNTuple with a backend that uses Intel DAOS as the underlying storage, demonstrating that the RNTuple architecture can accommodate high-performance object stores. From the user perspective, data can be accessed with minimal changes to the code, that is by replacing a filesystem path by a DAOS URI. Our performance evaluation shows that the new backend can be used for realistic analyses, while outperforming the compatibility solution provided by the DAOS project.