Automatic berthing using supervised learning and reinforcement learning

• Published in: Ocean engineering 2022-12, Vol.265, p.112553, Article 112553
• Main Authors: Shimizu, Shoma, Nishihara, Kenta, Miyauchi, Yoshiki, Wakita, Kouki, Suyama, Rin, Maki, Atsuo, Shirakawa, Shinichi
• Format: Article
• Language: English
• Subjects: Automatic berthing/docking; Autonomous vessel; Reinforcement learning; Supervised learning
• ISSN: 0029-8018; 1873-5258
• DOI: 10.1016/j.oceaneng.2022.112553

Although various studies have been conducted on automatic berthing, including offline optimization and online control, real-time berthing control remains a difficult problem. Online control methods without reference trajectories are promising for real-time berthing control. We used reinforcement learning (RL), which is a type of machine learning, to obtain an online control law without reference trajectories. As online control for automatic berthing is difficult, obtaining an appropriate control law with naive reinforcement learning is difficult. Furthermore, almost all existing online control methods do not consider port geometries. This study proposes a method for obtaining online berthing control laws by combining supervised learning (SL) and RL. We first trained the controller using offline-calculated trajectories and then further trained it using RL. Owing to the SL process, the proposed method can start the RL process with a good control policy. We evaluated the control law performance of the proposed method in a simulation environment that considered port geometries and wind disturbances. The experimental results show that the proposed method can achieve a higher success rate and lower safety risk than the naive SL and RL algorithms. •Obtained an online control law using supervised learning and reinforcement learning.•Showed a high success rate and low safety risk by simulation experiment.•Enabled online control considering port geometries without reference trajectories.