Quality-Oriented Hybrid Path Planning Based on A and Q-Learning for Unmanned Aerial Vehicle

• Published in: IEEE access 2022, Vol.10, p.7664-7674
• Main Authors: Li, Dongcheng, Yin, Wangping, Wong, W. Eric, Jian, Mingyong, Chau, Matthew
• Format: Article
• Language: English
• Subjects: A algorithm; Algorithms; Autonomous aerial vehicles; Cost function; Costs; Exploration; Heuristic algorithms; hierarchical planning; Machine learning; Maneuverability; Operating costs; Path planning; Planning; Q-learning; quality-oriented path planning; reinforcement learning; Safety; Turning; Unmanned aerial vehicle; Unmanned aerial vehicles
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2021.3139534

Unmanned aerial vehicles (UAVs) are playing an increasingly important role in people's daily lives due to their low cost of operation, low requirements for ground support, high maneuverability, high environmental adaptability, and high safety. Yet UAV path planning under various safety risks, such as crash and collision, is not an easy task, due to the complicated and dynamic nature of path environments. Therefore, developing an efficient and flexible algorithm for UAV path planning has become inevitable. Aimed at quality-oriented UAV path planning, this paper is designed to analyze UAV path planning from two aspects: global static planning and local dynamic hierarchical planning. Through a theoretical and mathematical approach, a three-dimensional UAV path planning model was established. Based on the A* algorithm, the search strategy, the step size, and the cost function were improved, and the OPEN set was simplified, thereby shortening the planning time and greatly improving the execution efficiency of the algorithm. Moreover, a dynamic exploration factor was added to the exploration mechanism of Q-learning to solve the exploration-exploitation dilemma of Q-learning to adapt to the local dynamic path adjustment for UAVs. The global-local hybrid UAV path planning algorithm was formed by combining the two. The simulation results indicate that the proposed planning model and algorithm can efficiently solve the problem of UAV path planning, improve the path quality, and can be a significant reference for solving other problems related to path planning, such as the reliability, security, and safety of UAV, when embedded into the heuristic function of the proposed algorithm.