MDAN: Multilevel dual-branch attention network for infrared and visible image fusion

• Published in: Optics and lasers in engineering 2024-05, Vol.176, p.108042, Article 108042
• Main Authors: Wang, Jiawei, Jiang, Min, Kong, Jun
• Format: Article
• Language: English
• Subjects: Attention mechanism; Deep learning; Fusion strategy; Infrared and visible image fusion; Singular value decomposition
• ISSN: 0143-8166; 1873-0302
• DOI: 10.1016/j.optlaseng.2024.108042

Infrared and visible image fusion (IVIF) aims to integrate information captured by optical sensors operating in two different modalities, generating a fused image with both salient targets and texture details. Despite significant advancements in IVIF algorithms, the challenge of preserving complete information, especially regarding texture details, still persists. To alleviate this problem, we propose a multilevel dual-branch attention network (MDAN) which comprises an encoder-decoder network and a fusion strategy layer composed of dual-branch fusion block (DBFB). Firstly, the encoder-decoder network is designed to extract multilevel image features and reconstruct the fused images. Secondly, a novel loss function based on singular value decomposition is proposed to constrain the reconstructed images to preserve abundant algebra features which reflect the structure and texture information of the source images. Thirdly, a fusion strategy layer based on spatial-channel attention and feature aggregation block, which consists of DBFB, is proposed to integrate the extracted features. Finally, we evaluate our method through qualitative and quantitative experiments, the results demonstrate that our method exhibits superiority in performance and achieves a remarkable balance between visual perception and objective evaluation metrics when compared to the state-of-the-art (SOTA) methods. •The Multilevel Dual-Branch Attention Network (MDAN) is proposed to extract multi-level features for generating fused images.•A fusion strategy layer based on spatial-channel attention and feature aggregation block is proposed to fused the extracted features.•A loss function based on singular value decomposition is designed to preserve the algebra features of the source images.•A large number of experiments prove the rationality and superiority of the proposed method.