Collaborative Human Recognition With Lightweight Models in Drone-Based Search and Rescue Operations

• Published in: IEEE transactions on vehicular technology 2024-02, Vol.73 (2), p.1765-1776
• Main Authors: Xu, Lijuan, Yang, Qinghai, Qin, Meng, Wu, Weihua, Kwak, KyungSup
• Format: Article
• Language: English
• Subjects: Accuracy; Algorithms; Artificial neural networks; Buffers; Cameras; Collaboration; collaborative recognition; Computational modeling; Convolutional neural networks; drone; Drones; Evacuations & rescues; Feature maps; human recognition; Image recognition; Lightweight; lightweight model; Machine learning; Mathematical models; Modules; Parameters; proximal policy optimization (PPO); Rescue operations; Search and rescue (SAR) operations; Search and rescue missions; Task analysis
• ISSN: 0018-9545; 1939-9359
• DOI: 10.1109/TVT.2023.3319483

Due to the flexibility of drones, it is promising to use them in Search and Rescue (SAR) operations for intelligently searching for lost humans over large areas. However, the limited computing resources of drones pose challenges when deploying deep neural network models. To address this problem, we design a lightweight human recognition model for drones, combining the Ghost Module and the Mobilenetv3 Block. The Ghost Module generates more feature maps with fewer parameters, and the squeeze-and-excitation (SE) attention module in the Mobilenetv3 Block greatly improves recognition accuracy. Furthermore, recognizing that relying solely on the human recognition model offers limited assistance in SAR operations, we propose a collaborative recognition mode between the drone and the rescue command center (RCC). In this collaborative recognition mode, we design an offloading model for deployment on the drone. The offloading model learns from the middle layer perception features of the lightweight recognition model and selectively offloads the vision taken by the drone to the RCC. The recognition results provided by the RCC are used to update the parameters of the offloading model. We introduce a reinforcement learning algorithm, a dual-buffer-based proximal policy optimization algorithm (DBPPO), to train the offloading model with the goal of maximizing accuracy and recall while minimizing the offloading ratio. Specifically, we incorporate an additional data buffer for training the Actor network in the PPO algorithm in a supervised manner, with supervised training interspersed throughout the PPO training process. Eventually, experiments comparing different methods demonstrate the effectiveness of the lightweight recognition model and the collaborative recognition mode in SAR operations.