Infrared image super-resolution method based on dual-branch deep neural network

• Published in: The Visual computer 2024-03, Vol.40 (3), p.1673-1684
• Main Authors: Zhijian, Huang, Bingwei, Hui, Shujin, Sun, Yanxin, Ma
• Format: Article
• Language: English
• Subjects: Artificial Intelligence; Artificial neural networks; Computer Graphics; Computer Science; Deep learning; Generative adversarial networks; Image contrast; Image enhancement; Image Processing and Computer Vision; Image quality; Image reconstruction; Image resolution; Infrared imagery; Infrared imaging systems; Methods; Neural networks; Original Article
• ISSN: 0178-2789; 1432-2315
• DOI: 10.1007/s00371-023-02878-y

Infrared image has lower resolution, lower contrast, and less detail than visible image, which causes its super-resolution (SR) more difficult than visible image. This paper presents an approach based on a deep neural network that comprises an image SR branch and a gradient SR branch to reconstruct high-quality SR image from single-frame infrared image. The image SR branch reconstructs the SR image from the initial low-resolution infrared image using a basic structure similar to the enhanced SR generative adversarial network (ESRGAN). The gradient SR branch removes haze, extracts the gradient map, and reconstructs the SR gradient map. To obtain more natural SR image, a fusion block based on attention mechanism is adopted between these branches. To preserve the geometric structure, gradient L1 loss and gradient GAN loss are defined and added. Experimental results on a public infrared image dataset demonstrate that, compared with the current SR methods, the proposed method is more natural and realistic, and can better preserve the structures.