Inpainting of Remote Sensing Sea Surface Temperature image with Multi-scale Physical Constraints

• Main Authors: Wei, Qichen, Zuo, Zijie, Nie, Jie, Du, Jiahao, Diao, Yaning, Ye, Min, Liang, Xinyue
• Format: Conference Proceeding
• Language: English
• Subjects: Data visualization; deep learning; generative adversarial network; Generative adversarial networks; Industries; multi-scale; Neural networks; physical constraint; Sea surface; sea surface temperature; Semantics; Sensors
• ISSN: 1945-788X
• DOI: 10.1109/ICME55011.2023.00091

Sea Surface Temperature (SST) is a significant environmental factor indicating marine revolutions, which is popularly applied in the meteorological forecasting and fishing industry. Due to the limited sensing ability and occlusion caused by clouds or ice, it is difficult to obtain complete SST data. Compared to traditional interpolation-based methods which refill missed data only referred to current SST data, inpainting-based methods have been carried out with the advantage of using historical SST images to train Generative adversarial Networks (GAN) by terms of considering SST data reconstruction task as an image inpainting task. However, different from common inpainting tasks constrained by semantics, the SST image is a scientific data visualization image without semantics but physical constraints. To address this problem, this paper proposes a multi-scale inpainting GAN-based neural networks to guarantee the physical constraint and realize reasonable SST image reconstruction. The proposed framework mainly contains two modules including the Average Estimation Module (AEM) to realize a global constraint so as not to generate excessive deviation, and the Multi-scale Anomaly Decouple Module (MSADM) to preserve data specificity of current SST image from well-designed multi-scale and decoupled perspectives. Finally, a post-fusion module concatenates the "average" and "specificity" features together to accomplish our multi-scale physical constraints SST image inpainting task. Sufficient experiments have been carried out to verify the effectiveness and physical consistency compared with prior SOTA methods applied to the public AVHRR Pathfinder SST dataset.