Infrared–Visible Image Fusion through Feature-Based Decomposition and Domain Normalization

• Published in: Remote sensing (Basel, Switzerland) Switzerland), 2024-03, Vol.16 (6), p.969
• Main Authors: Chen, Weiyi, Miao, Lingjuan, Wang, Yuhao, Zhou, Zhiqiang, Qiao, Yajun
• Format: Article
• Language: English
• Subjects: Algorithms; Computer vision; Decomposition; Deep learning; dense attention; dynamic instance normalization; Feature decomposition; Feature extraction; Image quality; infrared and visible image fusion; Infrared imagery; Methods; non-local Gaussian filter; R&D; Radiation; Research & development; unified feature space
• ISSN: 2072-4292; 2072-4292
• DOI: 10.3390/rs16060969

Infrared–visible image fusion is valuable across various applications due to the complementary information that it provides. However, the current fusion methods face challenges in achieving high-quality fused images. This paper identifies a limitation in the existing fusion framework that affects the fusion quality: modal differences between infrared and visible images are often overlooked, resulting in the poor fusion of the two modalities. This limitation implies that features from different sources may not be consistently fused, which can impact the quality of the fusion results. Therefore, we propose a framework that utilizes feature-based decomposition and domain normalization. This decomposition method separates infrared and visible images into common and unique regions. To reduce modal differences while retaining unique information from the source images, we apply domain normalization to the common regions within the unified feature space. This space can transform infrared features into a pseudo-visible domain, ensuring that all features are fused within the same domain and minimizing the impact of modal differences during the fusion process. Noise in the source images adversely affects the fused images, compromising the overall fusion performance. Thus, we propose the non-local Gaussian filter. This filter can learn the shape and parameters of its filtering kernel based on the image features, effectively removing noise while preserving details. Additionally, we propose a novel dense attention in the feature extraction module, enabling the network to understand and leverage inter-layer information. Our experiments demonstrate a marked improvement in fusion quality with our proposed method.