Deep reinforcement learning and ant colony optimization supporting multi‐UGV path planning and task assignment in 3D environments

• Published in: IET intelligent transport systems 2024-09, Vol.18 (9), p.1652-1664
• Main Authors: Jin, Binghui, Sun, Yang, Wu, Wenjun, Gao, Qiang, Si, Pengbo
• Format: Article
• Language: English
• Subjects: ant colony optimization; deep reinforcement learning; multiple unmanned ground vehicles; path planning; task assignment
• ISSN: 1751-956X; 1751-9578
• DOI: 10.1049/itr2.12535

With the development of artificial intelligence, the application of unmanned ground vehicles (UGV) in outdoor hazardous scenarios has received more attention. However, the terrains in these environments are often complex and undulating, which also pose higher challenges to the multi‐UGV path planning and task assignment (MUPPTA) optimization. To efficiently improve the multi‐UGV collaboration in 3D environments, a MUPPTA method is proposed based on double deep Q learning network (DDQN) and ant colony optimization (ACO) to jointly optimize the path planning and task assignment decisions of multiple UGVs. The authors first comprehensively consider the characteristics of the 3D environments, and model the MUPPTA problem as a combinatorial optimization problem. To tackle it, the original problem is decomposed into the multi‐UGV path planning sub‐problem and task assignment sub‐problem, and solve them separately. First, the path planning sub‐problem in the 3D environments is transformed into a Markov decision process (MDP) model, and a multi‐UGV path planning algorithm based on DDQN (MUPP‐DDQN) is proposed to obtain the optimal paths and actual path costs between tasks through extensive offline learning and training. Based on this, a multi‐UGV task assignment algorithm is further proposed based on ACO (MUTA‐ACO) to solve the task assignment sub‐problem and achieve the optimal task assignment solution. Simulation results show that the proposed method is more cost‐effective and time‐saving compared to other comparison algorithms. This paper focus on the multi‐UGV path planning and task assignment (MUPPTA) problem in 3D environments, and propose a multi‐UGV path planning and task assignment method based on double DQN and ACO. Specifically, the algorithm takes the complex terrain and actual cost in 3D environments into consideration, and an optimization mechanism for multi‐UGV path planning and task assignment is established to guide the multi‐UGV coordination and reduce the system costs.