Fusion Network for Super Resolution of UAVs Visible and Thermal Images

• Main Authors: Fkih, Hedi, Delleji, Tijeni, Kallel, Abedelaziz, Chtourou, Zied
• Format: Conference Proceeding
• Language: English
• Subjects: Attention network; Convolution Networks; Feature extraction; Image resolution; Lighting; Measurement; Object recognition; Reference Image; Spatial fusion; Superresolution; Visualization
• ISSN: 2687-878X
• DOI: 10.1109/ATSIP55956.2022.9805951

Object identification is a fundamental and important problem in computer vision. Although impressive results have been obtained for medium to large objects in visible images, the performance on small objects, including mini/micro-UAVs (Unmanned Air Vehicle) in visible and thermal images, has received less attention. Therefore, due to the different appearances of these small flying objects, the identification of these objects, especially under poor early illumination conditions is currently a challenge. To overcome this challenge, we propose an increase in the resolution of the captured images by adding high frequency details which allows to produce better visual quality. In this work, we propose a Fusion Network for Super Resolution of UAVs Visible and Thermal Images (FNSR), which aims to produce a Hight Resolution image (HR) using visible and infrared (IR) reference images (Ref). FNSR is made up of five related modules, including a multi-scale texture extractor for feature extraction from visible and thermal images, a spatial fusion module, a relevance embedding module, a features transfer block and a spatial decoder module. We introduce channel attention mechanisms to highlight the most promising features, as well as a Spatial Total Variation (STV) loss to preserve spatial information. Experimental results on image pairs (IR/Visible) show that the FNSR is capable of producing high quality super-resolved images. Compared to the State-Of-The-Art (SOTA), it presents better performances.