Spectral Synthesis for Geostationary Satellite-to-Satellite Translation

• Published in: IEEE transactions on geoscience and remote sensing 2022-01, Vol.60, p.1-11
• Main Authors: Vandal, Thomas J., McDuff, Daniel, Wang, Weile, Duffy, Kate, Michaelis, Andrew, Nemani, Ramakrishna R.
• Format: Article
• Language: English
• Subjects: Atmospheric measurements; Band spectra; Biomonitoring; Domains; Earth; Earth Resources And Remote Sensing; Generative adversarial networks; Geophysical image processing; Image reconstruction; Imagery; Monitoring; NASA; neural networks (NNs); Oceans; Remote sensing; Satellite observation; Satellites; Sensors; Spectral bands; Synchronism; Synchronization; Synchronous satellites; Translation; unsupervised learning
• ISSN: 0196-2892; 1558-0644
• DOI: 10.1109/TGRS.2021.3088686

Earth-observing satellites carrying multispectral sensors are widely used to monitor the physical and biological states of the atmosphere, land, and oceans. These satellites have different vantage points above the Earth and different spectral imaging bands resulting in inconsistent imagery from one to another. This presents challenges in building downstream applications. What if we could generate synthetic bands for existing satellites from the union of all domains? We tackle the problem of generating synthetic spectral imagery for multispectral sensors as an unsupervised image-to-image translation problem modeled with a variational autoencoder (VAE) and generative adversarial network (GAN) architecture. Our approach introduces a novel shared spectral reconstruction loss to constrain the high-dimensional feature space of multispectral images. Simulated experiments performed by dropping one or more spectral bands show that cross-domain reconstruction outperforms measurements obtained from a second vantage point. Our proposed approach enables the synchronization of multispectral data and provides a basis for more homogeneous remote sensing datasets.