ICSF: Integrating Inter-Modal and Cross-Modal Learning Framework for Self-Supervised Heterogeneous Change Detection

• Published in: IEEE transactions on geoscience and remote sensing 2025, Vol.63, p.1-16
• Main Authors: Zhang, Erlei, Zong, He, Li, Xinyu, Feng, Mingchen, Ren, Jinchang
• Format: Article
• Language: English
• Subjects: Accuracy; Change detection; Contrastive learning; Deep learning; Disaster management; dual-branch; Environmental monitoring; Feature extraction; Geographical locations; heterogeneous change detection (HCD); Image processing; Image reconstruction; Image segmentation; Image sensors; Information processing; Learning; Nearest neighbor methods; Radar polarimetry; Remote monitoring; Remote sensing; self-supervised learning; Sensors; Sensory integration; Spatial data; Spatial discrimination learning; structural relationship
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2024.3519195

Heterogeneous change detection (HCD) is a process to determine the change information by analyzing heterogeneous images of the same geographic location taken at different times, which plays an important role in remote sensing applications such as disaster response and environmental monitoring. However, the different imaging mechanisms result in different visual appearances in heterogeneous images, making it difficult to accurately detect changes through direct comparison. To address this problem, we propose a inter-modal and cross-modal self-supervised dual branch learning framework (ICSF) for HCD that incorporates inter-modal and cross-modal learning. First, in the inter-modal branch, we perform contrastive learning on heterogeneous images within their respective modalities to learn the robust and discriminative features, rather than relying on the raw spectral or spatial information from these images. Second, in the cross-modal branch, we perform cross-modal reconstruction to ensure the obtained features exhibit consistent comparability, thereby facilitating the extraction of rich information on the real changes within the images. Next, the difference images (DIs) computed from both branches are further refined using a superpixel segmentation strategy to preserve the consistency of differences within the same ground object. Experimental results on five public datasets with different modality combinations and change events demonstrate the effectiveness of the proposed approach in comparison to ten state-of-the-art (SOTA) methods, achieving the best performance with an average overall accuracy (OA) of 95.88% and an average Kappa coefficient (KC) of 74.20%.