Multi-Drone Collaborative Shepherding Through Multi-Task Reinforcement Learning

• Published in: IEEE robotics and automation letters 2024-11, Vol.9 (11), p.10311-10318
• Main Authors: Wang, Guanghui, Peng, Junkun, Guan, Chenyang, Chen, Jinhua, Guo, Bing
• Format: Article
• Language: English
• Subjects: Adaptation models; Biological system modeling; Collaboration; Deep reinforcement learning; Drones; Heuristic algorithms; Multitasking; Path planning for multiple mobile robots or agents; reinforcement learning; shepherding
• ISSN: 2377-3766; 2377-3766
• DOI: 10.1109/LRA.2024.3468155

Robotic shepherding has become indispensable in animal husbandry and crowd management, offering a modern solution to traditional challenges. Drone Automated Shepherding leverages advanced maneuverability and an extensive field of view to improve the efficiency of these tasks, which are typically labor-intensive and time-consuming. Existing methods for managing large herds face significant challenges due to insufficient coordination among multiple drones and the complexities involved in simultaneously executing diverse shepherding tasks. This paper aims to enhance the execution of multiple shepherding tasks by optimizing drone coordination, designing optimal flight paths, and reducing flight time. To harness the potential of reinforcement learning, we develop a multi-drone collaborative shepherding environment that facilitates efficient drone training using a dense reward system. Additionally, we employ a multi-task deep reinforcement learning algorithm that enhances the sample efficiency and reward performance by leveraging shared information across tasks within this environment. Two specific tasks, driving and collecting, are used to assess the performance of our methodology. The effectiveness of our approach is measured against a classical solution named CTRL, examining metrics such as success rate, completion time, and flight path length. Results indicate that our approach significantly outperforms the CTRL in all measured metrics. Visualization of drone trajectories provides further evidence of our enhanced collaboration and efficiency in shepherding operations. Real-world experiments are conducted on a square of 300 \text{m}^{3}, where two drones utilize our method to guide four small autonomous vehicles from the starting area to the goal area within 30 seconds.