Provenance-based Workflow Diagnostics Using Program Specification

Workflow management systems (WMS) help automate and coordinate scientific modules and monitor their execution. WMSes are also used to repeat a workflow application with different inputs to test sensitivity and reproducibility of runs. However, when differences arise in outputs across runs, current W...

Full description

Saved in:
Bibliographic Details
Main Authors: Nakamura, Yuta, Malik, Tanu, Kanj, Iyad, Gehani, Ashish
Format: Conference Proceeding
Language:English
Subjects:
Debugging
High performance computing
Metadata
provenance alignment
Reproducibility of results
Scientific computing
Sensitivity
Workflow management software
workflow management systems
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Workflow management systems (WMS) help automate and coordinate scientific modules and monitor their execution. WMSes are also used to repeat a workflow application with different inputs to test sensitivity and reproducibility of runs. However, when differences arise in outputs across runs, current WMSes do not audit sufficient provenance metadata to determine where the execution first differed. This increases diagnostic time and leads to poor quality diagnostic results. In this paper, we use program specification to precisely determine locations where workflow execution differs. We use existing provenance audited to isolate modules where execution differs. We show that using program specification comes at some increased storage overhead due to mapping of provenance data flows onto program specification, but leads to better quality diagnostics in terms of the number of differences found and their location relative to comparing provenance metadata audited within current WMSes.
ISSN:2640-0316
DOI:10.1109/HiPC56025.2022.00046