DDM Workload Emulation

Rucio is the successor of the current Don Quijote 2 (DQ2) system for the distributed data management (DDM) system of the ATLAS experiment. The reasons for replacing DQ2 are manifold, but besides high maintenance costs and architectural limitations, scalability concerns are on top of the list. Curren...

Full description

Saved in:
Bibliographic Details
Published in:Journal of physics. Conference series 2014-01, Vol.513 (4), p.42048-5
Main Authors: Vigne, R, Schikuta, E, Garonne, V, Stewart, G, Barisits, M, Beermann, T, Lassnig, M, Serfon, C, Goossens, L, Nairz, A
Format: Article
Language:English
Subjects:
Availability
Catalogues
Data management
Lists
Logs
Maintenance costs
Manifolds
Physics
Provisioning
Pushing
Workload
Workloads
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Rucio is the successor of the current Don Quijote 2 (DQ2) system for the distributed data management (DDM) system of the ATLAS experiment. The reasons for replacing DQ2 are manifold, but besides high maintenance costs and architectural limitations, scalability concerns are on top of the list. Current expectations are that the amount of data will be three to four times as it is today by the end of 2014. Further is the availability of more powerful computing resources pushing additional pressure on the DDM system as it increases the demands on data provisioning. Although DQ2 is capable of handling the current workload, it is already at its limits. To ensure that Rucio will be up to the expected workload, a way to emulate it is needed. To do so, first the current workload, observed in DQ2, must be understood in order to scale it up to future expectations. The paper discusses how selected core concepts are applied to the workload of the experiment and how knowledge about the current workload is derived from various sources (e.g. analysing the central file catalogue logs). Finally a description of the implemented emulation framework, used for stress-testing Rucio, is given.
ISSN:1742-6588
1742-6596
DOI:10.1088/1742-6596/513/4/042048