Attention mechanism-based generative adversarial networks for cloud removal in Landsat images

• Published in: Remote sensing of environment 2022-03, Vol.271, p.112902, Article 112902
• Main Authors: Xu, Meng, Deng, Furong, Jia, Sen, Jia, Xiuping, Plaza, Antonio J.
• Format: Article
• Language: English
• Subjects: Attention mechanism; Cloud removal; Clouds; Deep learning; Feature extraction; Feature maps; Generative adversarial networks; Generative adversarial networks (GANs); Image quality; Image restoration; Landsat; Landsat images; Remote sensing; Satellite imagery
• ISSN: 0034-4257; 1879-0704
• DOI: 10.1016/j.rse.2022.112902

The existence of clouds affects the quality of optical remote sensing images. Cloud removal is an important preprocessing procedure to effectively improve the utilization of optical remote sensing images. Thin clouds partly obscure the land surfaces beneath them, making it possible to correct the cloudy scenes according to the available information. In this research, we introduce the attention mechanism-based generative adversarial networks for cloud removal (AMGAN-CR) method for Landsat images. First, attention maps of the input cloudy images are generated to extract the cloud distributions and features through an attentive recurrent network. Second, clouds are removed by an attentive residual network under the guidance of the attention maps. Finally, the generated feature maps are fed to a reconstruction network to restore the final cloud-free images. The networks are trained by cloudy and cloud-free Landsat image pairs, and the cloudy images are tested to validate the effectiveness of AMGAN-CR. Both simulated and real cloud experimental results show that the proposed method is more outstanding than the other five state-of-the-art traditional and deep learning methods in removing cloud. •Generative adversarial networks (GANs) is applied for cloud removal.•We use attention mechanism to capture the distribution of cloud thickness.•Cloud detection is not required in AMGAN-CR.•Take advantage of multispectral information to remove clouds.•The new method performs better than the other five state-of-the-art methods.