Transfer Learning Based Multi-Objective Evolutionary Algorithm for Dynamic Workflow Scheduling in the Cloud

Managing scientific applications in the Cloud poses many challenges in terms of workflow scheduling, especially in handling multi-objective workflow scheduling under quality of service (QoS) constraints. However, most studies address the workflow scheduling problem on the premise of the unchanged en...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on cloud computing 2024-01, Vol.12 (4), p.1200-1217
Main Authors: Xie, Huamao, Ding, Ding, Zhao, Lihong, Kang, Kaixuan
Format: Article
Language:English
Subjects:
Cloud computing
Constraints
Costs
Dynamic scheduling
Dynamic workflow scheduling
Energy consumption
Evolutionary algorithms
Genetic algorithms
Heuristic algorithms
Knowledge management
Machine learning
multi-objective evolutionary algorithm
Multiple objective analysis
Optimization
preference optimization
Processor scheduling
Quality of service
quality of service constraints
Scheduling
Strategy
transfer learning
Workflow
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Managing scientific applications in the Cloud poses many challenges in terms of workflow scheduling, especially in handling multi-objective workflow scheduling under quality of service (QoS) constraints. However, most studies address the workflow scheduling problem on the premise of the unchanged environment, without considering the high dynamics of the Cloud. In this paper, we model the constrained workflow scheduling in a dynamic Cloud environment as a dynamic multi-objective optimization problem with preferences, and propose a transfer learning based multi-objective evolutionary algorithm (TL-MOEA) to tackle the workflow scheduling problem of dynamic nature. Specifically, an elite-led transfer learning strategy is proposed to explore effective parameter adaptation for the MOEA by transferring helpful knowledge from elite solutions in the past environment to accelerate the optimization process. In addition, a multi-space diversity learning strategy is developed to maintain the diversity of the population. To satisfy various QoS constraints of workflow scheduling, a preference-based selection strategy is further designed to enable promising solutions for each iteration. Extensive experiments on five well-known scientific workflows demonstrate that TL-MOEA can achieve highly competitive performance compared to several state-of-art algorithms, and can obtain triple win solutions with optimization objectives of minimizing makespan, cost and energy consumption for dynamic workflow scheduling with user-defined constraints.
ISSN:2168-7161
2372-0018
DOI:10.1109/TCC.2024.3450858