Domain knowledge integration into deep learning for typhoon intensity classification

• Published in: Scientific reports 2021-06, Vol.11 (1), p.12972-12972, Article 12972
• Main Authors: Higa, Maiki, Tanahara, Shinya, Adachi, Yoshitaka, Ishiki, Natsumi, Nakama, Shin, Yamada, Hiroyuki, Ito, Kosuke, Kitamoto, Asanobu, Miyata, Ryota
• Format: Article
• Language: English
• Subjects: 639/705/1042; 639/705/117; 704/106; Classification; Deep learning; Embedding; Eye; Humanities and Social Sciences; Integration; multidisciplinary; Science; Science (multidisciplinary); Stochasticity; Typhoons
• ISSN: 2045-2322; 2045-2322
• DOI: 10.1038/s41598-021-92286-w

In this report, we propose a deep learning technique for high-accuracy estimation of the intensity class of a typhoon from a single satellite image, by incorporating meteorological domain knowledge. By using the Visual Geometric Group’s model, VGG-16, with images preprocessed with fisheye distortion, which enhances a typhoon’s eye, eyewall, and cloud distribution, we achieved much higher classification accuracy than that of a previous study, even with sequential-split validation. Through comparison of t-distributed stochastic neighbor embedding (t-SNE) plots for the feature maps of VGG with the original satellite images, we also verified that the fisheye preprocessing facilitated cluster formation, suggesting that our model could successfully extract image features related to the typhoon intensity class. Moreover, gradient-weighted class activation mapping (Grad-CAM) was applied to highlight the eye and the cloud distributions surrounding the eye, which are important regions for intensity classification; the results suggest that our model qualitatively gained a viewpoint similar to that of domain experts. A series of analyses revealed that the data-driven approach using only deep learning has limitations, and the integration of domain knowledge could bring new breakthroughs.