Online Multi-Workflow Scheduling under Uncertain Task Execution Time in IaaS Clouds

• Published in: IEEE transactions on cloud computing 2021-07, Vol.9 (3), p.1180-1194
• Main Authors: Liu, Jiagang, Ren, Ju, Dai, Wei, Zhang, Deyu, Zhou, Pude, Zhang, Yaoxue, Min, Geyong, Najjari, Noushin
• Format: Article
• Language: English
• Subjects: Algorithms; Cloud computing; Heuristic methods; multiple workflows; Pricing; Processor scheduling; Resource management; Resource utilization; Schedules; Scheduling; Stochastic processes; Task analysis; Task scheduling; uncertain task execution time; Virtual environments; VM rental cost; Workflow; workflow scheduling
• ISSN: 2168-7161; 2168-7161; 2372-0018
• DOI: 10.1109/TCC.2019.2906300

Cloud has become an important platform for executing numerous deadline-constrained scientific applications generally represented by workflow models. It provides scientists a simple and cost-efficient method of running workflows on their rental Virtual Machines (VMs) anytime and anywhere. Since pay-as-you-go is a dominating pricing solution in clouds, extensive research efforts have been devoted to minimizing the monetary cost of executing workflows by designing tailored VM allocation mechanisms. However, most of them assume that the task execution time in clouds is static and can be estimated in advance, which is impractical in real scenarios due to performance fluctuation of VMs. In this paper, we propose an onli N e multi-workfl O w S cheduling F ramework, named NOSF, to schedule deadline-constrained workflows with random arrivals and uncertain task execution time. In NOSF, workflow scheduling process consists of three phases, including workflow preprocessing, VM allocation and feedback process. Built upon the new framework, a deadline-aware heuristic algorithm is then developed to elastically provision suitable VMs for workflow execution, with the objective of minimizing the rental cost and improving resource utilization. Simulation results demonstrate that the proposed algorithm significantly outperforms two state-of-the-art algorithms in terms of reducing VM rental costs and deadline violation probability, as well as improving the resource utilization efficiency.