Lightweight unmanned aerial vehicle video object detection based on spatial‐temporal correlation

• Published in: International journal of communication systems 2022-11, Vol.35 (17), p.n/a
• Main Authors: Zhou, Pei, Liu, GuanJun, Wang, Jiacun, Weng, QianLi, Zhang, KaiWen, Zhou, ZiYuan
• Format: Article
• Language: English
• Subjects: computing capacity; Context; Deep learning; Edge computing; Image processing; Machine learning; Modules; Navigation; Object recognition; Power consumption; spatial‐temporal correlation; UAV; Unmanned aerial vehicles; video object detection
• ISSN: 1074-5351; 1099-1131
• DOI: 10.1002/dac.5334

Summary Intelligent unmanned aerial vehicles (UAVs) are drawing more and more attention from industry to academia. UAV navigation plays an important role in the cooperative scenario where multiple UAVs are deployed, while image data that capture the information of the UAV area are often used as input for UAV navigation. Deep learning is a common and powerful technique for UAV image processing, but a complex model generated by deep learning technique is hardly suitable for the limited computing capacity of edge computing devices such as UAVs. Therefore, this paper designs an efficient deep learning model on UAVs to fit the restriction of low computational powers and low power consumption. Traditional UAV object detection methods mostly use static images as the basis for object recognition, or collect images for offline detection. Our method combines the existing fast single‐frame detection methods with the spatial‐temporal relationship of video sequences, to build an efficient end‐to‐end model. In addition, the convolutional LSTM module is used to propagate the temporal context of the video frame sequences. Based on the temporal context, we propose a module for calculating spatial correlation. At the same time, we establish our experimental dataset in our real application and conduct the experiment, which shows that the proposed method reduces the size of models and meanwhile maintains the detection rate. Compared with the existing static images approaches, our method is faster and more accurate. Inference speeds of nearly 20fps can be achieved while performing real‐time tasks. This paper designs a lightweight UAV video object detection model based on spatial‐temporal correlation to deal with the restriction of low computational power and low power consumption of UAVs. The spatial‐temporal correlation module is proposed to propagate the context of the UAV video frame sequences, which can improve the pixel‐level representation ability of feature graphs and the final detection performance. While performing the UAV real‐time object detection tasks, this work has achieved the inference speed of nearly 20 FPS and the recognition rate of 95%.