Multi-objective Reinforcement Learning for Responsive Grids

• Published in: Journal of grid computing 2010-09, Vol.8 (3), p.473-492
• Main Authors: Perez, Julien, Germain-Renaud, Cécile, Kégl, Balazs, Loomis, Charles
• Format: Article
• Language: English
• Subjects: Approximation; Computer Science; Distributed, Parallel, and Cluster Computing; Elasticity; Feedback loops; Learning; Machine Learning; Management of Computing and Information Systems; Mathematical analysis; Mathematical models; Modeling and Simulation; Multiple objective analysis; Processor Architectures; Provisioning; Reinforcement; Resource allocation; Strategy; User Interfaces and Human Computer Interaction; User satisfaction
• ISSN: 1570-7873; 1572-9184
• DOI: 10.1007/s10723-010-9161-0

Grids organize resource sharing, a fundamental requirement of large scientific collaborations. Seamless integration of Grids into everyday use requires responsiveness, which can be provided by elastic Clouds, in the Infrastructure as a Service (IaaS) paradigm. This paper proposes a model-free resource provisioning strategy supporting both requirements. Provisioning is modeled as a continuous action-state space, multi-objective reinforcement learning (RL) problem, under realistic hypotheses; simple utility functions capture the high level goals of users, administrators, and shareholders. The model-free approach falls under the general program of autonomic computing, where the incremental learning of the value function associated with the RL model provides the so-called feedback loop. The RL model includes an approximation of the value function through an Echo State Network. Experimental validation on a real data-set from the EGEE Grid shows that introducing a moderate level of elasticity is critical to ensure a high level of user satisfaction.