Hypergraph Self-Supervised Learning-Based Joint Spectral-Spatial-Temporal Feature Representation for Hyperspectral Image Change Detection

• Published in: IEEE journal of selected topics in applied earth observations and remote sensing 2025-01, Vol.18, p.741-756
• Main Authors: Jian, Ping, Ou, Yimin, Chen, Keming
• Format: Article
• Language: English
• Subjects: Contrastive learning; Correlation; Data models; Feature extraction; generative learning; hypergraph model; hyperspectral image change detection (HSI-CD); Image reconstruction; Manuals; Mathematical models; Principal component analysis; Representation learning; self-supervised learning (SSL); spectral-spatial-temporal feature representation; Transformers
• ISSN: 1939-1404; 2151-1535
• DOI: 10.1109/JSTARS.2024.3483560

Deep learning has shown promising performance in the field of hyperspectral image (HSI) change detection (CD). However, most of these methods often focus on local spatial-spectral information but ignore the high-order correlations contained in multitemporal HSIs. To address this issue, this article proposes a hypergraph self-supervised learning (HG-SSL) based joint spectral-spatial-temporal feature representation algorithm (HyperSST) for downstream HSI-CD. Inspired by the process of human brain perception, HyperSST uniformly models the spectral-spatial-temporal correlations in the form of high-order interactions and skillfully exploits the vertex-level, hyperedge-level, and vertex/hyperedge-level inherent structures within the unlabeled multitemporal HSIs. Specifically, two types of hyperedges, spectral-spatial correlation hyperedge and temporal correlation hyperedge, are first formulated to fully exploit the high-order spectral, spatial, and temporal interactions contained in bitemporal images. Second, two SSL strategies namely contrastive spectral-spatial features learning and generative temporal features learning are skillfully designed to exploit the inherent characteristics of hypergraph models and extract discriminative latent feature representations for downstream tasks. The former captures changes in both attribute space and structure space, while the latter apprehends the changes in temporal space. Third, the learned joint spectral-spatial-temporal features provide a comprehensive representation to qualify the changes between multitemporal images. Extensive experiments on four challenging HSI datasets demonstrate the effectiveness of the proposed approach.