Adaptive VNF Placement Considering Overall Latency and 5G Wireless Channel Reliability in Industry 4.0: A Reinforcement Learning Based Approach

• Published in: IEEE access 2024, Vol.12, p.88883-88896
• Main Authors: Saqib, Nauman, Abdullah, Nor Fadzilah, Abu-Samah, Asma, Alobaidy, Haider A. H., Nordin, Rosdiadee
• Format: Article
• Language: English
• Subjects: 5G mobile communication; Algorithms; Automation; Cloud computing; Deep reinforcement learning; Delay; Fourth Industrial Revolution; Industrial applications; Industry 4.0; Internet of Things; Iterative algorithms; Machine learning; Markov processes; Millimeter wave communication; Millimeter waves; mmWave; Network function virtualization; Network latency; Optimization; Placement; Production facilities; Reliability; URLLC; Virtual networks; VNF placement; Wireless communication; Wireless sensor networks
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2024.3419065

Industry 4.0 incorporates the integration of cloud computing, Industrial Internet of Things (IIoT), and modern communication technologies within the industrial automation systems. Various devices with different network requirements of high reliability and low latency, rely on connectivity. The 5G and Beyond (B5G) software-defined architecture facilitates Network Function Virtualization (NFV), which is an essential solution for fulfilling these stringent demands. NFV allows for the implementation and control of Virtual Network Functions (VNFs) in dynamic network environments. VNF placement optimization has been extensively studied in the 5G perspective outside the industry environment with a focus on minimizing delay and cost, increasing VNF reliability, and increasing resource efficiency. However, the complex dynamics of the wireless channel in industrial environments have a considerable impact on the essential delay factors that are important for optimizing the deployment of VNFs. This study focuses on modeling a Wireless Sensor Network (WSN) based Industry 4.0 factory automation scenario at mmWave band, formulating an optimization problem to minimize overall delay while considering packet loss rate in the 5G industrial wireless channel. The optimization problem is formulated as a Markov Decision Process (MDP) and two Reinforcement Learning (RL) based algorithms AVP-Q and AVP-DQN are proposed for optimizing the VNF placement. The proposed algorithms are extensively evaluated against the Value Iteration algorithm which assumes a completely known MDP model and two other algorithms from the literature. The simulated results show that AVP-DQN outperforms existing algorithms for this scenario by 39% and 22.6% and the achieved performance is only close to that of the Value Iteration algorithm.