Eye in the Sky: Energy Efficient Model-Based Reinforcement Learning Aided Target Tracking Using UAVs

• Published in: IEEE transactions on vehicular technology 2024-01, Vol.73 (12), p.19464-19479
• Main Authors: Xia, Yi, Zhang, Zekai, Xu, Jingzehua, Ren, Pengfei, Wang, Jingjing, Han, Zhu
• Format: Article
• Language: English
• Subjects: Algorithms; Autonomous aerial vehicles; Constraints; Control systems design; Convergence; Cost effectiveness; Costs; Deep reinforcement learning; Energy consumption; mobile target tracking; Reconnaissance aircraft; Robust control; Search and rescue missions; Steady state models; Target tracking; time constraints; Time factors; Tracking; UAV; Unmanned aerial vehicles; Vehicle dynamics
• ISSN: 0018-9545; 1939-9359
• DOI: 10.1109/TVT.2024.3437776

The rapid response and high energy efficiency of the unmanned aerial vehicle (UAV) are crucial prerequisites for enabling time-sensitive and long-endurance target tracking missions, such as search and rescue, area reconnaissance, and convoy monitoring. However, existing research in target tracking primarily focuses on enhancing tracking accuracy, which struggles to adapt to tasks considering strict time constraints and energy consumption. To address these issues, this paper introduces a model-based reinforcement learning tracking strategy (MRLTS) for the UAV to minimize control costs and achieve user-specified tracking performance, including a two-stage design. In the first stage, a steady-state robust tracking controller is developed based on available model knowledge that forces the UAV to asymptotically approximate a predefined observation path in spite of uncertainties. In the second stage, an intelligent component based on the soft actor-critic (SAC) algorithm is customized to empower the UAV to strike a trade-off between prescribed tracking performance and energy consumption, wherein a skilled barrier function is constructed to interpret specified time constraints. The proposed paradigm can provide a higher sampling efficiency than SAC-based strategy. Simulation results demonstrate that our strategy outperforms benchmarks and results in a 46.3% cost-effectiveness improvement at least.