Spatiotemporal Reflectance Fusion via Tensor Sparse Representation

• Published in: IEEE transactions on geoscience and remote sensing 2022, Vol.60, p.1-18
• Main Authors: Peng, Yidong, Li, Weisheng, Luo, Xiaobo, Du, Jiao, Zhang, Xiayan, Gan, Yi, Gao, Xinbo
• Format: Article
• Language: English
• Subjects: Catchment area; Coefficients; Computer vision; Correlation; Cubes; Degradation; Dictionaries; Dictionary learning; Earth surface; Exploitation; Feasibility studies; Glossaries; Image processing; Instruments; Learning; Learning systems; Mapping; Methods; Neglect syndromes; nonlocal tensor sparse representation (TSR); Reflectance; Remote sensing; Representations; Resolution; Satellite instruments; Satellite-borne instruments; Sensors; Similarity; Spatial data; Spatial discrimination; Spatial resolution; spatial-spectral-nonlocal correlation; spatiotemporal fusion; Spatiotemporal phenomena; Spectra; Spectral correlation; Surface dynamics; Tensors
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2021.3091157

Tradeoffs between the spatial and temporal resolutions of current satellite instruments limit our ability to conduct high-quality and continuous monitoring of the earth's surface dynamics. Spatiotemporal image fusion has become increasingly necessary to obtain remote sensing images with high spatiotemporal resolution. However, current learning-based methods concentrate on predicting images only from spatial similarity and neglect spectral correlations of remote sensing images, leading to significant spectral information loss. In this article, we develop a novel nonlocal tensor sparse representation-based semicoupled dictionary learning approach (SCDNTSR) for spatiotemporal fusion. In the SCDNTSR method, the spectral correlation and the spatial similarity of the nonlocal similar cubes are simultaneously exploited through the tensor-tensor product-based tensor sparse representation. Furthermore, the semicoupled mapping prior knowledge of sparse coefficients across the high- and low-spatial resolution (HSR\LSR) image spaces is exploited with the coupled dictionary to constrain the similarity of sparse coefficients to improve the prediction performance. In addition, to capture additional prior spatial information, the SCDNTSR provides a new method to determine the degradation relationship between the target HSR and LSR difference images with the help of the known HSR and LSR difference images. The proposed SCDNTSR method was tested on real datasets at both the Coleambally Irrigation Area study site and the Lower Gwydir Catchment study site. Results show that the proposed method outperforms five state-of-the-art methods, especially in maintaining the spectral information, proving the feasibility of integrating the degradation relationship, spatio-spectral-nonlocal correlation, and semicoupled mapping priors of the multisource data into the proposed model.