Hybrid Gradient Descent Golden Eagle Optimization (HGDGEO) Algorithm-Based Efficient Heterogeneous Resource Scheduling for Big Data Processing on Clouds

• Published in: Wireless personal communications 2023-03, Vol.129 (2), p.1175-1195
• Main Authors: Jagadish Kumar, N., Balasubramanian, C.
• Format: Article
• Language: English
• Subjects: Algorithms; Big Data; Cloud computing; Communications Engineering; Computer Communication Networks; Data processing; Data transmission; Decision making; Dynamic characteristics; Engineering; Networks; Optimization; Resource allocation; Resource scheduling; Resource utilization; Scheduling; Signal,Image and Speech Processing; Strategy
• ISSN: 0929-6212; 1572-834X
• DOI: 10.1007/s11277-023-10182-0

Resource scheduling is indispensable for enhancing the system performance during big data processing on clouds. It is highly useful for attaining significant utilization of computing resources completely concentrating towards the facilitation of resource scalability and on-demand services. The resources essential for running different applications is determined to be maximum heterogeneous in cloud computing. This heterogeneous resource demand introduces a resource gap in which some of the resource potentialities are drained on par with the other resource potentialities still available in the same server resulting in imbalanced resource utilization. This imbalanced resource allocation condition is more apparent when the computing resources are more heterogeneous. At this juncture, intelligent resource scheduling strategy becomes essential to distribute resources for big data processing by adopting a potential decision-making process that focusses on the objective of achieving necessitated tasks over time. In this paper, Hybrid Gradient Descent Golden Eagle Optimization (HGDGEO) algorithm-based efficient heterogeneous resource scheduling process is proposed for handling the challenges that are highly possible during big data processing in the Hadoop heterogenous cloud environment. This HGDGEO algorithm is proposed as an adaptive resource scheduling strategy that handles the dynamic characteristics of the resources and users’ fluctuating demand during big data stream processing by mimicking the golden eagles’ intelligence which alternates the speed of tuning at different spiral trajectory stages of hunting. It handles big data processing by adopting two adaptive parameters which completely concentrates on optimal resource allocation to suitable VMs in the shortest time possible depending on their requirements. The simulation results of HGDGEO algorithm confirmed its predominance in terms of makespan, load balance and throughput on par with the competitive resource scheduling algorithms.