Fault Tolerance Oriented SFC Optimization in SDN/NFV-Enabled Cloud Environment Based on Deep Reinforcement Learning

• Published in: IEEE transactions on cloud computing 2024-01, Vol.12 (1), p.200-218
• Main Authors: Chen, Jing, Chen, Jia, Guo, Kuo, Hu, Renkun, Zou, Tao, Zhu, Jun, Zhang, Hongke, Liu, Jingjing
• Format: Article
• Language: English
• Subjects: Cloud computing; Constraint modelling; Costs; Deep learning; deep reinforcement learning; Design optimization; elastic optimization; Energy consumption; Fault tolerance; fault-tolerant; Heuristic algorithms; Integer programming; Load modeling; Markov processes; Multiagent systems; Network function virtualization; Optimization; Quality of service; Servers; Service function chain; Software-defined networking; Upper bounds; Virtual networks
• ISSN: 2168-7161; 2372-0018
• DOI: 10.1109/TCC.2024.3357061

In software defined network/network function virtualization (SDN/NFV)-enabled cloud environment, cloud services can be implemented as service function chains (SFCs), which consist of a series of ordered virtual network functions. However, due to fluctuations of cloud traffic and without knowledge of cloud computing network configuration, designing SFC optimization approach to obtain flexible cloud services in dynamic cloud environment is a pivotal challenge. In this paper, we propose a fault tolerance oriented SFC optimization approach based on deep reinforcement learning. We model fault tolerance oriented SFC elastic optimization problem as a Markov decision process, in which the reward is modeled as a weighted function, including minimizing energy consumption and migration cost, maximizing revenue benefit and load balancing. Then, taking binary integer programming model as constraints of quality of cloud services, we design optimization approaches for single-agent double deep Q-network (SADDQN) and multi-agent DDQN (MADDQN). Among them, MADDQN decentralizes training tasks from control plane to data plane to reduce the probability of single point of failure for the centralized controller. Experimental results show that the designed approaches have better performance. MADDQN can almost reach the upper bound of theoretical solution obtained by assuming a prior knowledge of the dynamics of cloud traffic.