A novel perception and semantic mapping method for robot autonomy in orchards

• Published in: Computers and electronics in agriculture 2024-04, Vol.219, p.108769, Article 108769
• Main Authors: Pan, Yaoqiang, Hu, Kewei, Cao, Hao, Kang, Hanwen, Wang, Xing
• Format: Article
• Language: English
• Subjects: 3D detection; Navigation; Robot autonomy; Semantic mapping
• ISSN: 0168-1699; 1872-7107
• DOI: 10.1016/j.compag.2024.108769

Agricultural robots must navigate challenging dynamic and semi-structured environments. Recently, environmental modelling using LiDAR-based SLAM has shown promise in providing highly accurate geometry. However, how this chaotic environmental information can be used to achieve effective robot automation in the agricultural sector remains unexplored. In this study, we propose a novel semantic mapping and navigation framework for achieving robotic autonomy in orchards. It consists of two main components: a semantic processing module and a navigation module. First, we present a novel 3D detection network architecture, 3D-ODN, which can accurately process object instance information from point clouds. Second, we develop a framework to construct the visibility map by incorporating semantic information and terrain analysis. By combining these two critical components, our framework is evaluated in a number of key horticultural production scenarios, including a robotic system for in-situ phenotyping and daily monitoring, and a selective harvesting system in apple orchards. The experimental results show that our method can ensure high accuracy in understanding the environment and enable reliable robot autonomy in agricultural environments. •Create a robust 3D Object Detection Network (3D-ODN) to process 3D point cloud from SLAM.•Develop a novel semantic mapping framework involving the 3D-ODN and terrain analysis.•Demonstrate the proposed perception and semantic mapping framework on mobile robots in orchards.