Air-to-ground multimodal object detection algorithm based on feature association learning

• Published in: International journal of advanced robotic systems 2019-05, Vol.16 (3)
• Main Authors: Yang, Dongfang, Liu, Xing, He, Hao, Li, Yongfei
• Format: Article
• Language: English
• Subjects: Algorithms; Cameras; Feature maps; Infrared cameras; Light; Machine learning; Object recognition; Target detection; Unmanned aerial vehicles; Weather
• ISSN: 1729-8806; 1729-8814
• DOI: 10.1177/1729881419842995

Detecting objects on unmanned aerial vehicles is a hard task, due to the long visual distance and the subsequent small size and lack of view. Besides, the traditional ground observation manners based on visible light camera are sensitive to brightness. This article aims to improve the target detection accuracy in various weather conditions, by using both visible light camera and infrared camera simultaneously. In this article, an association network of multimodal feature maps on the same scene is used to design an object detection algorithm, which is the so-called feature association learning method. In addition, this article collects a new cross-modal detection data set and proposes a cross-modal object detection algorithm based on visible light and infrared observations. The experimental results show that the algorithm improves the detection accuracy of small objects in the air-to-ground view. The multimodal joint detection network can overcome the influence of illumination in different weather conditions, which provides a new detection means and ideas for the space-based unmanned platform to the small object detection task.