A balanced and reliable data replica placement scheme based on reinforcement learning in edge–cloud environments

• Published in: Future generation computer systems 2024-06, Vol.155, p.132-145
• Main Authors: Zheng, Mengke, Du, Xin, Lu, Zhihui, Duan, Qiang
• Format: Article
• Language: English
• Subjects: Data reliability; Heterogeneous edge–cloud; Load balancing; Reinforcement learning; Replica placement
• ISSN: 0167-739X; 1872-7115
• DOI: 10.1016/j.future.2024.02.004

With the rapid development of edge–cloud computing, distributing resources to edge nodes and terminal devices to provide high-quality services for latency-sensitive applications and reduce network communication costs has become increasingly important. However, the complexity and heterogeneity of the edge–cloud environment pose significant challenges to the reliability of data storage and device load balancing. To address these issues, in this paper, we propose a Deep Reinforcement Learning (DRL)-based data replica placement scheme, BRPS. This scheme considers the different geographic locations of hardware devices, the heterogeneous storage capacity and reliability, and the different data requirements of varying user services in edge–cloud environments. Firstly, we constructed a model for data replica placement in the edge–cloud environment, addressing key factors such as latency, reliability, and load, focusing on the heterogeneity of device resources and reliability, and the diverse data needs of user services. Furthermore, we propose the BRPS scheme using the Double Deep Q-Network (DDQN) method of DRL, transforming the data replica placement issue into a multi-objective optimization problem. By placing the DRL-based decision process in a separate management edge node, the separation of decision and execution is achieved, which enhances the efficiency of data replica placement, ensures data reliability, reduces latency, and achieves system load balance. Experimental results demonstrate that BRPS significantly outperforms the existing comparison schemes while ensuring data reliability. The BRPS scheme reduces latency by 8.12% compared to the Random scheme and outperforms the best heuristic in system load balancing and memory utilization by 13.15% and 11.91%, respectively. Moreover, BRPS shows superior adaptability in extreme network congestion scenarios and effectively adapts to the dynamic changes of nodes in edge–cloud environments. •We propose a data replica placement model in a hybrid edge-cloud architecture based on real scenarios.•We design the data replica placement scheme, BRPS, based on DRL under this model. We transformed the data replication placement problem into a multi-objective optimization problem.•We implement the separation of decision and execution by centralizing the DRL-based decision in the management edge node.