Navigation path recognition between rows of fruit trees based on semantic segmentation

• Published in: Computers and electronics in agriculture 2024-01, Vol.216, p.108511, Article 108511
• Main Authors: Zhang, Liang, Li, Ming, Zhu, Xinghui, Chen, Yedong, Huang, Jinqi, Wang, Zhiwei, Hu, Tian, Wang, Ziru, Fang, Kui
• Format: Article
• Language: English
• Subjects: agricultural machinery and equipment; agriculture; data collection; electronics; fruits; Image processing; kiwifruit; Navigation; Orchard; orchards; Path recognition; peaches; prediction; Semantic segmentation; Transfer learning
• ISSN: 0168-1699; 1872-7107
• DOI: 10.1016/j.compag.2023.108511

•We propose Fast-Unet based on Unet pruning and optimization to recognize navigation paths between rows of peach, orange and kiwifruit.•We propose to use ASPP for feature extraction in model down-sampling and Bilinear interpolation for up-sampling to improve model inference speed and segmentation accuracy.•We adopt the loss function Dice Loss that can optimize the intersection of union and the BCE Loss that can improve the accuracy of difficult classification categories as the loss function of the model.•We use transfer learning to transfer models trained on a large sample peach dataset to small sample datasets of orange and kiwifruit, verify the feasibility of expanding the use of transfer learning through small samples, and propose a generalized method for navigation path recognition between rows of fruit trees.•We extract the path boundaries using the Canny operator and fit the navigation lines using the least square method. The navigation path recognition has been recognized as one of the most important subtasks of intelligent agricultural equipment in orchard operations. However, there are still some challenges in recognizing navigation paths between rows of fruit trees, including the accuracy, real-time performance, generalization of deep learning models. The Fast-Unet model was proposed by pruning and optimization based on Unet for recognizing navigation paths between rows of fruit trees, which inherited encoding–decoding structure and multi-layer feature sensing capability. The number of convolutional kernels used to extract features in the Fast-Unet was reduced to one-fourth of that in Unet to improve inference speed. To address the blurring of the boundary of the recognized object due to the reduction in the number of convolutional kernels, the atrous spatial pyramid pooling (ASPP) module was used in the encoding part to extract the multiscale information to improve the recognition accuracy. The navigation path edges determined by Fast-Unet and Canny operators, navigation lines and yaw angles were generated by the least square method.. In this study, the Fast-Unet model was first trained on the peach dataset, and then the trained model was transferred to the small dataset of oranges and kiwifruits for navigation path recognition to verify the generalization. The Mean Intersection over Union (MIOU) of the Fast-Unet for peaches, oranges and kiwifruits navigation path extraction accuracy were 0.977, 0.987 and 0.956, respectively. The mean difference