Spatial-Spectral Interaction Super-Resolution CNN-Mamba Network for Fusion of Satellite Hyperspectral and Multispectral Image

• Published in: IEEE journal of selected topics in applied earth observations and remote sensing 2024, Vol.17, p.18489-18501
• Main Authors: Zhao, Guangwei, Wu, Haitao, Luo, Dexiang, Ou, Xu, Zhang, Yu
• Format: Article
• Language: English
• Subjects: Convolutional neural network (CNN); Data mining; Distortion; Feature extraction; fusion; hyperspectral; Image fusion; Image reconstruction; Mamba; multispectral; Spatial resolution; Superresolution
• ISSN: 1939-1404; 2151-1535
• DOI: 10.1109/JSTARS.2024.3469184

The tradeoff between spatial and spectral resolution in sensor design is inevitable, and spatial-spectral fusion aims to use low spatial resolution hyperspectral image (HSI) and high spatial resolution (HR) multispectral image (MSI) obtained at the same time and in the same area to reconstruct HR HSI. Recently, a large number of deep-learning methods have been applied in this field and achieved success. However, these methods do not fully utilize the characteristics of data for network design, and cannot guarantee effective computational efficiency in extracting local and global features. To solve the above problems, we designed a spatial-spectral interaction super-resolution convolutional neural network (CNN)-Mamba fusion network for satellite HSI and MSI, which uses mutual guidance to improve the spatial and spectral resolution of different data, and obtains the final fused image through feature fusion. In addition, we combined Mamba with CNN to effectively explore global and local features of images. Extensive experiments have proven that our method can reconstruct fused images of high quality and is superior to current state-of-the-art fusion methods.