FogFusionNet: Coastal Sea Fog Prediction by Using a Multimodal Deep Learning Approach

• Published in: IEEE access 2024, Vol.12, p.137491-137503
• Main Authors: Son, Doo-Hwan, Chun, Minki, Kim, Young Taeg, Kang, Boonsoon, Kim, Kuk Jin, Han, Jin Hyun
• Format: Article
• Language: English
• Subjects: Atmospheric measurements; Closed circuit television; Closed-circuit television (CCTV) images; coastal sea fog prediction; Coastal waters; Coastlines; Data models; Economic impact; Feature extraction; Intervals; Low visibility; multimodal learning; Multivariate analysis; multivariate time series observation data (MTSO); Numerical models; Ocean temperature; Performance prediction; Prediction models; Predictive models; Sea measurements; Time series analysis; visibility class
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2024.3401179

In this study, we designed FogFusionNet, a multimodal sea fog prediction model, that used closed-circuit television (CCTV) images and multivariate time series observation (MTSO) data to predict three visibility classes-Normal visibility, Low visibility, and Sea fog-at 1-h intervals from the current time to 6-h in the future for a specific region. We applied weighted sampling and weighted loss to overcome the imbalance of each visibility class, and additionally evaluated the effect of replacing missing MTSO data. A total of 4 years of data regarding Incheon Port, which faces the Yellow Sea and is prone to sea fog, were collected for training and verifying FogFusionNet. Of these, 3 years of data was used for training FogFusionNet, and the remaining 1 year of data were used for verifying the performance of FogFusionNet. The prediction performance of FogFusionNet at 1-h intervals was 86.2% (0-h), 79.1% (1-h), 73.4% (2-h), 70.7% (3-h), 64.7% (4-h), 59.6% (5-h), and 49.3% (6-h), showing an average prediction performance of 69.0%. FogFusioneNet is expected to promote coastal safety and reduce economic losses due to coastal sea fog.