Parallel Placement of Virtualized Network Functions via Federated Deep Reinforcement Learning

• Published in: IEEE/ACM transactions on networking 2024-08, Vol.32 (4), p.2936-2949
• Main Authors: Huang, Haojun, Tian, Jialin, Min, Geyong, Yin, Hao, Zeng, Cheng, Zhao, Yangming, Wu, Dapeng Oliver
• Format: Article
• Language: English
• Subjects: Convergence; Costs; Deep reinforcement learning; federated learning; Hidden Markov models; multiple domains; Network function virtualization; parallel placement; Parallel processing; Servers; Training
• ISSN: 1063-6692; 1558-2566
• DOI: 10.1109/TNET.2024.3366950

Network Function Virtualization (NFV) introduces a new network architecture that offers different network services flexibly and dynamically in the form of Service Function Chains (SFCs), which refer to a set of Virtualization Network Functions (VNFs) chained in a specific order. However, the service latency often increases linearly with the length of SFCs due to the sequential execution of VNFs, resulting in sub-optimal performance for most delay-sensitive applications. In this paper, a novel Parallel VNF Placement (PVFP) approach is proposed for real-world networks via Federated Deep Reinforcement Learning (FDRL). PVFP has three remarkable characteristics distinguishing from previous work: 1) PVFP designs a specific parallel principle, with three parallelism identification rules, to reasonably decide partial VNF parallelism; 2) PVFP considers SFC partition in multi-domains built on their remaining resources and potential parallel VNFs to ensure that VNFs can be reasonably distributed for resource balancing among domains; 3) FDRL-based framework of parallel VNF placement is designed to train a global intelligent model, with time-variant local autonomy explorations, for cross-domain SFC deployment, avoiding data sharing among domains. Simulation results in different scenarios demonstrate that PVFP can significantly reduce the end-to-end latency of SFCs at the medium resource expenditures to place VNFs in multiple administrative domains, compared with the state-of-the-art mechanisms.