HAM-MFN: Hyperspectral and Multispectral Image Multiscale Fusion Network With RAP Loss

• Published in: IEEE transactions on geoscience and remote sensing 2020-07, Vol.58 (7), p.4618-4628
• Main Authors: Xu, Shuang, Amira, Ouafa, Liu, Junmin, Zhang, Chun-Xia, Zhang, Jiangshe, Li, Guanghai
• Format: Article
• Language: English
• Subjects: Angle loss; convolutional neural network (CNN); Distortion; Feature extraction; hyperspectral image (HSI); Hyperspectral imaging; image fusion; Image processing; Laplace equations; Laplacian loss; Machine learning; multispectral image (MSI); Neural networks; Remote sensing; Spatial resolution; Tensors
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2020.2964777

The fusion of hyperspectral image (HSI) and multispectral image (MSI) is one of the most significant topics in remote sensing image processing. Recently, deep learning (DL) has emerged as an important tool for this task. However, existing DL-based methods have two drawbacks, that is, limited ability for feature extraction and suffering from spectral distortion. To address these issues, this article presents a novel neural network, where sophisticated techniques are employed, including network-in-network convolutional unit, batch normalization, and skip connection. To make full use of the MSI, the proposed model fuses HSI and MSI at different scales. Besides, this article presents a new loss function, called RMSE, angle and Laplacian (RAP) loss (the combination of the relative mean squared error, angle loss, and Laplacian loss), to deal with both spatial and spectral distortions. Experiments conducted on four data sets have verified the rationality of network structure and the proposed loss function and demonstrated that the proposed novel model outperforms state-of-the-art counterparts.