SSCAConv: Self-guided Spatial-Channel Adaptive Convolution for Image Fusion

• Published in: IEEE geoscience and remote sensing letters 2024-01, Vol.21, p.1-1
• Main Authors: Lu, Xiaoya, Zhuo, Yu-Wei, Chen, Hongming, Deng, Liang-Jian, Hou, Junming
• Format: Article
• Language: English
• Subjects: Algorithms; Computer vision; Convolution; Convolutional Neural Network; Hyperspectral image super-resolution; Hyperspectral imaging; Image Fusion; Image processing; Image resolution; Pansharpening; Pluto; Remote Sensing; Specificity
• ISSN: 1545-598X; 1558-0571
• DOI: 10.1109/LGRS.2023.3344944

Pansharpening, which attempts to obtain a high-resolution multispectral (HR-MS) image by fusing a panchromatic (PAN) image with a low-resolution multispectral (LR-MS) image, is a critical yet difficult remote sensing image processing task. In this study, we present a novel convolution operation, self-guided spatial-channel adaptive convolution (SSCAConv), for pansharpening. Unlike the reported adaptive convolutions that only focus on spatial details, our SSCAConv also considers channel specificity by generating an individual convolution kernel for each channel patch according to its own content and supplements the inter-channel information by introducing a global bias. We further apply the designed SSCAConv to a simple residual network architecture to construct the image fusion network (SSCANet). Experimental results show that SSCANet outperforms state-of-the-art (SOTA) pansharpening algorithms and achieves better generalization ability with fewer parameters. Additionally, our network also yields the best results when extended to the hyperspectral image super-resolution (HISR) problem. The code is available at https://github.com/Pluto-wei/SSCAConv.