Metrics for heterogeneous scientific workflows: A case study of an earthquake science application

Scientific workflows are a common computational model for performing scientific simulations. They may include many jobs, many scientific codes, and many file dependencies. Since scientific workflow applications may include both high-performance computing (HPC) and high-throughput computing (HTC) job...

Full description

Saved in:
Bibliographic Details
Published in:The international journal of high performance computing applications 2011-08, Vol.25 (3), p.274-285
Main Authors: Callaghan, Scott, Maechling, Philip, Small, Patrick, Milner, Kevin, Juve, Gideon, Jordan, Thomas H, Deelman, Ewa, Mehta, Gaurang, Vahi, Karan, Gunter, Dan, Beattie, Keith, Brooks, Christopher
Format: Article
Language:English
Subjects:
Business metrics
Computation
Computational efficiency
Earthquakes
Employment
Floating point arithmetic
High performance computing
Mathematical models
Optimization
Seismic phenomena
Studies
Tasks
Workflow
Workflow software
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Scientific workflows are a common computational model for performing scientific simulations. They may include many jobs, many scientific codes, and many file dependencies. Since scientific workflow applications may include both high-performance computing (HPC) and high-throughput computing (HTC) jobs, meaningful performance metrics are difficult to define, as neither traditional HPC metrics nor HTC metrics fully capture the extent of the application. We describe and propose the use of alternative metrics to accurately capture the scale of scientific workflows and quantify their efficiency. In this paper, we present several specific practical scientific workflow performance metrics and discuss these metrics in the context of a large-scale scientific workflow application, the Southern California Earthquake Center CyberShake 1.0 Map calculation. Our metrics reflect both computational performance, such as floating-point operations and file access, and workflow performance, such as job and task scheduling and execution. We break down performance into three levels of granularity: the task, the workflow, and the application levels, presenting a complete view of application performance. We show how our proposed metrics can be used to compare multiple invocations of the same application, as well as executions of heterogeneous applications, quantifying the amount of work performed and the efficiency of the work. Finally, we analyze CyberShake using our proposed metrics to determine potential application optimizations.
ISSN:1094-3420
1741-2846
DOI:10.1177/1094342011414743