Method for Identifying Litchi Picking Position Based on YOLOv5 and PSPNet

• Published in: Remote sensing (Basel, Switzerland) Switzerland), 2022-05, Vol.14 (9), p.2004
• Main Authors: Qi, Xiaokang, Dong, Jingshi, Lan, Yubin, Zhu, Hang
• Format: Article
• Language: English
• Subjects: Accuracy; Algorithms; Citrus fruits; Clustering; Computer architecture; Deep learning; Fruits; Image processing; Image segmentation; litchi; Localization; Machine learning; Neural networks; Path planning; Picking; picking point; Pixels; PSPNet; Remote sensing; Research methodology; Robots; Semantic segmentation; Stems; Target detection; Wavelet transforms; Workflow; YOLOv5
• ISSN: 2072-4292; 2072-4292
• DOI: 10.3390/rs14092004

China has the largest output of litchi in the world. However, at present, litchi is mainly picked manually, fruit farmers have high labor intensity and low efficiency. This means the intelligent unmanned picking system has broad prospects. The precise location of the main stem picking point of litchi is very important for the path planning of an unmanned system. Some researchers have identified the fruit and branches of litchi; however, there is relatively little research on the location of the main stem picking point of litchi. So, this paper presents a new open-access workflow for detecting accurate picking locations on the main stems and presents data used in the case study. At the same time, this paper also compares several different network architectures for main stem detection and segmentation and selects YOLOv5 and PSPNet as the most promising models for main stem detection and segmentation tasks, respectively. The workflow combines deep learning and traditional image processing algorithms to calculate the accurate location information of litchi main stem picking points in the litchi image. This workflow takes YOLOv5 as the target detection model to detect the litchi main stem in the litchi image, then extracts the detected region of interest (ROI) of the litchi main stem, uses PSPNet semantic segmentation model to semantically segment the ROI image of the main stem, carries out image post-processing operation on the ROI image of the main stem after semantic segmentation, and obtains the pixel coordinates of picking points in the ROI image of the main stem. After coordinate conversion, the pixel coordinates of the main stem picking points of the original litchi image are obtained, and the picking points are drawn on the litchi image. At present, the workflow can obtain the accurate position information of the main stem picking point in the litchi image. The recall and precision of this method were 76.29% and 92.50%, respectively, which lays a foundation for the subsequent work of obtaining the three-dimensional coordinates of the main stem picking point according to the image depth information, even though we have not done this work in this paper.