GEO-WMS: an improved approach to geoscientific workflow management system on HPC

With the increasing complexity of scientific computing, it is imperative to enhance the efficiency and ease of High Performance Computing (HPC) utilization. Scientific workflow is introduced to that aim, but the current infrastructure still needs optimization. In this paper, we discuss the current p...

Full description

Saved in:
Bibliographic Details
Published in:CCF transactions on high performance computing (Online) 2023-12, Vol.5 (4), p.360-373
Main Authors: Guo, Jiaxu, Xu, Yidan, Fu, Haohuan, Xue, Wei, Gan, Lin, Tan, Mengxuan, Wu, Tingye, Shen, Yutong, Wu, Xianwei, Hu, Liang, Che, Xilong
Format: Article
Language:English
Subjects:
Advances in Developing Large Scale Earth System Models
Complexity
Computer Hardware
Computer Science
Computer Systems Organization and Communication Networks
Datasets
Earth science
Efficiency
Experiments
Regular Paper
Researchers
Scientists
Simulation
Supercomputers
Teams
User experience
User interfaces
Workflow management systems
Workflow software
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:With the increasing complexity of scientific computing, it is imperative to enhance the efficiency and ease of High Performance Computing (HPC) utilization. Scientific workflow is introduced to that aim, but the current infrastructure still needs optimization. In this paper, we discuss the current problems based on scientific computing scenarios and design a more user-friendly workflow system solution targeting HPC services. In the proposed solution, we introduce a structured method to describe the workflow and employ a more user-friendly interface for scientific workflows to bring a better experience than traditional command line approaches. We have integrated a variety of methods to enhance the user experience during geoscience experiments. Data analytics are being used to make more intelligent recommendations to users. Runtime predictions help users to better plan their schedules for research. The statistics of the testing period and user feedback show that the proposed workflow management system can effectively save the operating time and complexity of the scientists, while saving computing resources. Our proposed system has a variety of advantageous features, including the ease of use, uniform specification with scalability, improved utilization of computing resources, and exemplary significance.
ISSN:2524-4922
2524-4930
DOI:10.1007/s42514-022-00131-x