Hybrid analysis pipelines in the REANA reproducible analysis platform

We introduce the feasibility of running hybrid analysis pipelines in the REANA reproducible analysis platform. The REANA platform allows researchers to specify declarative computational workflow steps describing the analysis process and to execute analysis workload on remote containerised compute cl...

Full description

Saved in:
Bibliographic Details
Main Authors: Rodríguez, Diego, Mačiulaitis, Rokas, Okraska, Jan, Šimko, Tibor
Format: Conference Proceeding
Language:English
Subjects:
Cloud computing
Control systems design
Particle physics
Pipelines
Workflow
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We introduce the feasibility of running hybrid analysis pipelines in the REANA reproducible analysis platform. The REANA platform allows researchers to specify declarative computational workflow steps describing the analysis process and to execute analysis workload on remote containerised compute clouds. We have designed an abstract job controller component permitting to execute different parts of the analysis workflow on different compute backends, such as HTCondor, Kubernetes and SLURM. We have prototyped the designed solution including the job execution, job monitoring, and input/output file staging mechanism between the various compute backends. We have tested the prototype using several particle physics model analyses. The present work introduces support for hybrid analysis workflows in the REANA reproducible analysis platform and paves the way towards studying underlying performance advantages and challenges associated with hybrid analysis patterns in complex particle physics data analyses.
ISSN:2100-014X
2101-6275
2100-014X
DOI:10.1051/epjconf/202024506041