Real-time significant wave height estimation from raw ocean images based on 2D and 3D deep neural networks

• Published in: Ocean engineering 2020-04, Vol.201, p.107129, Article 107129
• Main Authors: Choi, Heejeong, Park, Minsik, Son, Gyubin, Jeong, Jaeyun, Park, Jaesun, Mo, Kyounghyun, Kang, Pilsung
• Format: Article
• Language: English
• Subjects: Convolutional long short-term memory; Convolutional neural network; Ocean–wave image processing; Real-time significant wave height estimation
• ISSN: 0029-8018; 1873-5258
• DOI: 10.1016/j.oceaneng.2020.107129

The fuel costs, which constitute the highest proportion of sailing costs, vary considerably depending on ocean condition although ships sail on the same route. Among various ocean conditions, a wave height is one of the most significant factors to be considered for economic routing which aims at reducing fuel expenses. In this study, we propose deep neural network based approaches for real-time significant wave height estimation from solely raw ocean images. First, we estimate significant wave height level from single ocean image. Convolutional neural network (CNN) based classification model is constructed by investigating the four CNN structures and two performance improvement methods. Second, we propose a regression model that estimates real-valued significant wave heights from sequential ocean images. This model is based on convolutional long short-term memory to extract spatio-temporal features from time-series images. Experimental results on National Data Buoy Center dataset showed that the proposed classification model yielded an accuracy of 84%. In addition, the proposed regression model yielded a mean squared error of 0.0177 on the proposed dataset, which consisted of serial ocean images captured from a container ship. •A proposed CNN-based significant wave height classification model yielded 84% of the classification accuracy.•A proposed ConvLSTM-based significant wave height regression model yielded a mean squared error of 0.0177.•Significant wave height is classified/estimated based only on ocean images.•End-to-end learning without any feature engineering becomes possible.