SCT-CR: A synergistic convolution-transformer modeling method using SAR-optical data fusion for cloud removal

•The SCT-CR model combining CNN and Transformer for remote sensing cloud removal.•Synergistic convolution module with cloud attention for SAR-optical fusion.•Transformer based robust global context intelligent integration module. Traditional CNNs struggle with SAR and optical image fusion cloud remo...

Full description

Saved in:
Bibliographic Details
Published in:International journal of applied earth observation and geoinformation 2024-06, Vol.130, p.103909, Article 103909
Main Authors: Ma, Jianshen, Chen, Yumin, Pan, Jun, Xu, Jiangong, Li, Zhanghui, Xu, Rui, Chen, Ruoxuan
Format: Article
Language:English
Subjects:
Cloud removal
Data fusion
SAR
Synergistic Convolution
Transformer
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:•The SCT-CR model combining CNN and Transformer for remote sensing cloud removal.•Synergistic convolution module with cloud attention for SAR-optical fusion.•Transformer based robust global context intelligent integration module. Traditional CNNs struggle with SAR and optical image fusion cloud removal due to SAR image noise, feature space differences and random cloud distribution. This often leads to blurred results with less texture information. This paper proposes a synergistic convolution-transformer cloud removal method (SCT-CR), which is based on a specially designed synergistic convolution module that enables the synergistic fusion of SAR and optical imagery. The proposed network employs a transformer module in the high-dimensional section to better perceive the contextual information of the image and achieve intelligent extraction of global image features. The proposed SCT-CR network successfully addresses the problem of image blur in generated images and makes full use of the texture information present in SAR images. The SCT-CR model is tested on the spectral properties and recovery of visual effects. The experimental results on public datasets SEN12MS-CR and LuojiaSET-OSFCR show that the proposed model has stable and optimal performance. On the SEN12MS-CR dataset, the proposed model improves the SSIM metrics by 15.7 %, 10.2 %, 4.9 %, and 0.5 % compared to the SAR2OPT, SarOptcAGN, DSen2-CR, and GLF-CR models, respectively. On the LuojiaSET-OSFCR dataset, it was improved by 20.0 %, 10.0 %, 6.6 %, and 1.9 %, respectively.
ISSN:1569-8432
DOI:10.1016/j.jag.2024.103909