Network Slice Reconfiguration by Exploiting Deep Reinforcement Learning With Large Action Space

• Published in: IEEE eTransactions on network and service management 2020-12, Vol.17 (4), p.2197-2211
• Main Authors: Wei, Fengsheng, Feng, Gang, Sun, Yao, Wang, Yatong, Qin, Shuang, Liang, Ying-Chang
• Format: Article
• Language: English
• Subjects: Algorithms; branch dueling q-network; Consumption; core network slicing; Deep learning; deep reinforcement learning; Intelligent networks; Machine learning; Network slice reconfiguration; Network slicing; Numerical models; Optimization; Performance evaluation; Reconfiguration; Resource management; Resource utilization; Substrates; Uncertainty
• ISSN: 1932-4537; 1932-4537
• DOI: 10.1109/TNSM.2020.3019248

It is widely acknowledged that network slicing can tackle the diverse usage scenarios and connectivity services that the 5G-and-beyond system needs to support. To guarantee performance isolation while maximizing network resource utilization under dynamic traffic load, network slice needs to be reconfigured adaptively. However, it is commonly believed that the fine-grained resource reconfiguration problem is intractable due to the extremely high computational complexity caused by numerous variables. In this article, we investigate the reconfiguration within a core network slice with aim of minimizing long-term resource consumption by exploiting Deep Reinforcement Learning (DRL). This problem is also intractable by using conventional Deep Q Network (DQN), as it has a multi-dimensional discrete action space which is difficult to explore efficiently. To address the curse of dimensionality, we propose to exploit Branching Dueling Q-network which incorporates the action branching architecture into DQN to drastically decrease the number of estimated actions. Based on the discrete BDQ network, we develop an intelligent network slice reconfiguration algorithm (INSRA). Extensive simulation experiments are conducted to evaluate the performance of INSRA and the numerical results reveal that INSRA can minimize the long-term resource consumption and achieve high resource efficiency compared with several benchmark algorithms.