Self-Adaptive Resource Management for Large-Scale Shared Clusters

• Published in: Journal of computer science and technology 2010-09, Vol.25 (5), p.945-957
• Main Author: 李研 陈峰宏 孙熙 周明辉 焦文品 曹尔刚 梅宏
• Format: Article
• Language: English
• Subjects: Algorithms; Artificial Intelligence; Clusters; Computer Science; Data Structures and Information Theory; Distributed processing; Dynamics; Employment; Human; Information Systems Applications (incl.Internet); Markets; Nodes; Placement; R&D; Regular Paper; Research & development; Resource allocation; Resource management; Resources management; Servers; Software; Software Engineering; Theory of Computation; Workload; Workloads; 工作负载; 应用程序; 快速反应能力; 程序运行; 自适应资源管理; 资源分配
• ISSN: 1000-9000; 1860-4749
• DOI: 10.1007/s11390-010-9379-0

In a shared cluster,each application runs on a subset of nodes and these subsets can overlap with one another. Resource management in such a cluster should adaptively change the application placement and workload assignment to satisfy the dynamic applications workloads and optimize the resource usage.This becomes a challenging problem with the cluster scale and application amount growing large.This paper proposes a novel self-adaptive resource management approach which is inspired from human market：the nodes trade their shares of applications＇ requests with others via auction and bidding to decide its own resource allocation and a global high-quality resource allocation is achieved as an emergent collective behavior of the market.Experimental results show that the proposed approach can ensure quick responsiveness, high scalability,and application prioritization in addition to managing the resources effectively.