QLGR: A Q-learning-based Geographic FANET Routing Algorithm Based on Multi-agent Reinforcement Learning

• Published in: KSII transactions on Internet and information systems 2021, 15(11), , pp.4244-4274
• Main Authors: Qiu, Xiulin, Xie, Yongsheng, Wang, Yinyin, Ye, Lei, Yang, Yuwang
• Format: Article
• Language: English
• Subjects: Ad hoc networks (Computer networks); Algorithms; Computer network protocols; Drone aircraft; Multi-agent systems; Reinforcement learning (Machine learning); 컴퓨터학
• ISSN: 1976-7277; 1976-7277
• DOI: 10.3837/tiis.2021.11.020

The utilization of UAVs in various fields has led to the development of flying ad hoc network (FANET) technology. In a network environment with highly dynamic topology and frequent link changes, the traditional routing technology of FANET cannot satisfy the new communication demands. Traditional routing algorithm, based on geographic location, can "fall" into a routing hole. In view of this problem, we propose a geolocation routing protocol based on multi-agent reinforcement learning, which decreases the packet loss rate and routing cost of the routing protocol. The protocol views each node as an intelligent agent and evaluates the value of its neighbor nodes through the local information. In the value function, nodes consider information such as link quality, residual energy and queue length, which reduces the possibility of a routing hole. The protocol uses global rewards to enable individual nodes to collaborate in transmitting data. The performance of the protocol is experimentally analyzed for UAVs under extreme conditions such as topology changes and energy constraints. Simulation results show that our proposed QLGR-S protocol has advantages in performance parameters such as throughput, end-to-end delay, and energy consumption compared with the traditional GPSR protocol. QLGR-S provides more reliable connectivity for UAV networking technology, safeguards the communication requirements between UAVs, and further promotes the development of UAV technology. Keywords: FANET, GPSR, dynamic environment, multi-agent reinforcement learning, local information.