Terrain classification using mars raw images based on deep learning algorithms with application to wheeled planetary rovers

• Published in: Journal of terramechanics 2023-08, Vol.108, p.33-38
• Main Authors: Guo, Junlong, Zhang, Xingyang, Dong, Yunpeng, Xue, Zhao, Huang, Bo
• Format: Article
• Language: English
• Subjects: Deep convolutional neural network; Mars raw images; Terrain classification; Wheeled planetary rover
• ISSN: 0022-4898; 1879-1204
• DOI: 10.1016/j.jterra.2023.04.002

•Develop a dataset of Mars scenes from numerous raw Mars images.•Compare performance of three common deep learning models to select the appropriate one.•Conduct transfer learning and data augmentation to enhance selected model’s generation capacity.•Deep learning can realize autonomous, real-time Martian terrain classification. Scene information plays a crucial role in motion control, attitude perception, and path planning for wheeled planetary rovers (WPRs). Terrain recognition is the fundamental component of scene recognition. Due to the rich information, visual sensors are usually used in terrain classification. However, teleoperation delay prevents WPRs from using visual information efficiently. End-to-end learning method of deep learning (DL) that does not need complex image preprocessing was proposed to deal with this issue. This paper first built a terrain dataset (consists of loose sand, bedrock, small rock, large rock, and outcrop) using real Mars images to directly support You Only Look Once (YOLOv5) to test its performance on terrain classification. Because the capability of end-to-end training scheme is positively correlated with dataset, the performance of YOLOv5 can be significantly improved by exploiting orders of magnitude more data. The best combination of hyperparameters and models was achieved by slightly tuning YOLOv5, and data augmentation was also applied to optimize its accuracy. Furthermore, its performance was compared with two other end-to-end network architectures. Deep learning algorithms can be used in the future planetary exploration missions, such as WPRs autonomy improvement, traversability analysis, and avoiding getting trapped.