Tree-Structured Neural Network for Hyperspectral Pansharpening

• Published in: IEEE journal of selected topics in applied earth observations and remote sensing 2024, Vol.17, p.2516-2530
• Main Authors: He, Lin, Ye, Hanghui, Xi, Dahan, Li, Jun, Plaza, Antonio, Zhang, Mei
• Format: Article
• Language: English
• Subjects: Adaptive data distribution; Artificial neural networks; convolutional neural network (CNN); Convolutional neural networks; dynamic enhancement; hierarchical tree; hyperspectral (HS) images; Information processing; Neural networks; Optimized production technology; Pansharpening; Routing; Spatial data; Spatial discrimination; Spatial resolution; Training; Trees; Vegetation
• ISSN: 1939-1404; 2151-1535
• DOI: 10.1109/JSTARS.2023.3344117

Hyperspectral (HS) pansharpening refers to fusing low spatial resolution HS (LRHS) images with the corresponding panchromatic (PAN) images to create high spatial resolution HS (HRHS) images. Most of the existing HS pansharpening methods overlook the spatial and spectral imbalance of the ground objects of different types in the observed scenes. To address the dilemma, in this article we develop a novel tree-structured neural network (Tree-SNet) to form an adaptive spatial-spectral processing for HS pansharpening. The Tree-SNet method maps a convolutional neural network (CNN) onto a hierarchical tree structure, where routing nodes automatically tune the data distributed to tree paths, which is adaptive to the local characteristics of the data, while spatial enhancement (SpatE) and spectral enhancement (SpecE) modules are dynamically performed in the tree paths to further strengthen the adaptive processing. The proposed Tree-SNet is evaluated on several datasets, and the experimental results verify its superiority.