Deadline-sensitive workflow orchestration without explicit resource control

• Published in: Journal of parallel and distributed computing 2011-03, Vol.71 (3), p.343-353
• Main Authors: Ramakrishnan, Lavanya, Chase, Jeffrey S., Gannon, Dennis, Nurmi, Daniel, Wolski, Rich
• Format: Article
• Language: English
• Subjects: Algorithms; Applied sciences; Cloud computing; Clouds; Computer networks; Computer science; control theory; systems; Computer systems and distributed systems. User interface; Control systems; Deadline-sensitive workflows; Distributed processing; Exact sciences and technology; Failure; Grid computing; Information systems. Data bases; Mathematical models; Memory organisation. Data processing; Orchestration; Software; Workflow; Workflow scheduling
• ISSN: 0743-7315; 1096-0848
• DOI: 10.1016/j.jpdc.2010.11.010

Deadline-sensitive workflows require careful coordination of user constraints with resource availability. Current distributed resource access models provide varying degrees of resource control: from limited or none in grid batch systems to explicit in cloud systems. Additionally applications experience variability due to competing user loads, performance variations, failures, etc. These variations impact the quality of service (QoS) that goes unaccounted for in planning strategies. In this paper we propose Workflow ORchestrator for Distributed Systems ( WORDS) architecture based on a least common denominator resource model that abstracts the differences and captures the QoS properties provided by grid and cloud systems. We investigate algorithms for effective orchestration (i.e., resource procurement and task mapping) for deadline-sensitive workflows atop the resource abstraction provided in WORDS. Our evaluation compares orchestration methodologies over TeraGrid and Amazon EC2 systems. Experimental results show that WORDS enables effective orchestration possible at reasonable costs on batch queue grid and cloud systems with or without explicit resource control. ► Separation of concerns between resources and application layers. ► Resource abstraction to represent resource properties. ► Probabilistic Quality of Service model. ► Workflow Orchestration for Deadline-Sensitive Workflows.