Changen2: Multi-Temporal Remote Sensing Generative Change Foundation Model

• Published in: IEEE transactions on pattern analysis and machine intelligence 2025-02, Vol.47 (2), p.725-741
• Main Authors: Zheng, Zhuo, Ermon, Stefano, Kim, Dongjun, Zhang, Liangpei, Zhong, Yanfei
• Format: Article
• Language: English
• Subjects: Buildings; Change data synthesis; Computational modeling; Data models; Earth; foundation model; generative model; Remote sensing; Semantics; Stochastic processes; Synthetic data; synthetic data pre-training; Three-dimensional displays; Time series analysis
• ISSN: 0162-8828; 1939-3539; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2024.3475824

Our understanding of the temporal dynamics of the Earth's surface has been significantly advanced by deep vision models, which often require a massive amount of labeled multi-temporal images for training. However, collecting, preprocessing, and annotating multi-temporal remote sensing images at scale is non-trivial since it is expensive and knowledge-intensive. In this paper, we present scalable multi-temporal change data generators based on generative models, which are cheap and automatic, alleviating these data problems. Our main idea is to simulate a stochastic change process over time. We describe the stochastic change process as a probabilistic graphical model, namely the generative probabilistic change model (GPCM), which factorizes the complex simulation problem into two more tractable sub-problems, i.e., condition-level change event simulation and image-level semantic change synthesis. To solve these two problems, we present Changen2, a GPCM implemented with a resolution-scalable diffusion transformer which can generate time series of remote sensing images and corresponding semantic and change labels from labeled and even unlabeled single-temporal images. Changen2 is a "generative change foundation model" that can be trained at scale via self-supervision, and is capable of producing change supervisory signals from unlabeled single-temporal images. Unlike existing "foundation models", our generative change foundation model synthesizes change data to train task-specific foundation models for change detection. The resulting model possesses inherent zero-shot change detection capabilities and excellent transferability. Comprehensive experiments suggest Changen2 has superior spatiotemporal scalability in data generation, e.g., Changen2 model trained on 256^{2} 2 pixel single-temporal images can yield time series of any length and resolutions of 1,024^{2} 2 pixels. Changen2 pre-trained models exhibit superior zero-shot performance (narrowing the performance gap to 3% on LEVIR-CD and approximately 10% on both S2Looking and SECOND, compared to fully supervised counterpart)