Infrared and Visible Image Fusion via Attention-Based Adaptive Feature Fusion

Infrared and visible image fusion methods based on feature decomposition are able to generate good fused images. However, most of them employ manually designed simple feature fusion strategies in the reconstruction stage, such as addition or concatenation fusion strategies. These strategies do not p...

Full description

Saved in:
Bibliographic Details
Published in:Entropy (Basel, Switzerland) Switzerland), 2023-02, Vol.25 (3), p.407
Main Authors: Wang, Lei, Hu, Ziming, Kong, Quan, Qi, Qian, Liao, Qing
Format: Article
Language:English
Subjects:
adaptive fusion strategy
Algorithms
attention mechanism
Blurring
Computer vision
Data fusion
Decomposition
Deep learning
Dictionaries
Feature decomposition
Generators
image fusion
Image processing
Image quality
Image reconstruction
Influence
Infrared imagery
Infrared radiation
Light
Methods
Neural networks
Synthesis
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Infrared and visible image fusion methods based on feature decomposition are able to generate good fused images. However, most of them employ manually designed simple feature fusion strategies in the reconstruction stage, such as addition or concatenation fusion strategies. These strategies do not pay attention to the relative importance between different features and thus may suffer from issues such as low-contrast, blurring results or information loss. To address this problem, we designed an adaptive fusion network to synthesize decoupled common structural features and distinct modal features under an attention-based adaptive fusion (AAF) strategy. The AAF module adaptively computes different weights assigned to different features according to their relative importance. Moreover, the structural features from different sources are also synthesized under the AAF strategy before reconstruction, to provide a more entire structure information. More important features are thus paid more attention to automatically and advantageous information contained in these features manifests itself more reasonably in the final fused images. Experiments on several datasets demonstrated an obvious improvement of image fusion quality using our method.
ISSN:1099-4300
1099-4300
DOI:10.3390/e25030407