Graph Convolutional Reinforcement Learning for Collaborative Queuing Agents

• Published in: IEEE eTransactions on network and service management 2023-06, Vol.20 (2), p.1363-1377
• Main Authors: Fawaz, Hassan, Lesca, Julien, Quang, Pham Tran Anh, Leguay, Jeremie, Zeghlache, Djamal, Medagliani, Paolo
• Format: Article
• Language: English
• Subjects: adaptive WFQ; Algorithms; Artificial Intelligence; Bronzes; Computer Science; Cooperation; Deep learning; deep reinforcement learning; Delays; DGN; MADQN; multi-agent systems; Multiagent systems; Networking and Internet Architecture; Proposals; Quality of service; Queueing; Queuing; Routing; Service level agreements; Silver; Smart queuing; Throughput
• ISSN: 1932-4537; 1932-4537
• DOI: 10.1109/TNSM.2022.3226605

This paper explores the use of multi-agent deep learning as well as learning to cooperate principles to meet strict service level agreements, in terms of throughput and end-to-end delay, for a set of classified network flows. We consider agents built on top of a weighted fair queuing algorithm that continuously set weights for three flow groups: gold, silver, and bronze. We rely on a novel graph-convolution based, multi-agent reinforcement learning approach known as DGN. As benchmarks, we propose centralized and distributed deep Q-network algorithms and evaluate their performances in different network, traffic, and routing scenarios, highlighting both the effectiveness of our proposals and the importance of agent cooperation. We show that our DGN-based approach meets stringent throughput and delay requirements across different scenarios, decreasing silver and bronze flow median waiting delays by more than 50 % and reducing the SLA violations of the latter by nearly 60 %, with respect to a classic priority queuing approach.