Multi-class detection of kiwifruit flower and its distribution identification in orchard based on YOLOv5l and Euclidean distance

• Published in: Computers and electronics in agriculture 2022-10, Vol.201, p.107342, Article 107342
• Main Authors: Li, Guo, Fu, Longsheng, Gao, Changqing, Fang, Wentai, Zhao, Guanao, Shi, Fuxi, Dhupia, Jaspreet, Zhao, Kegang, Li, Rui, Cui, Yongjie
• Format: Article
• Language: English
• Subjects: Flower distribution; Flower phenology; Kiwifruit flower; Robotic pollination; YOLOv5l
• ISSN: 0168-1699; 1872-7107
• DOI: 10.1016/j.compag.2022.107342

•Selecting suitable flower for pollination based on flowers detection and distribution.•Kiwifruit flowers were labeled into 10 classes based on flower phenology.•Flower cluster and branch junction were applied for obtaining flower distribution.•YOLOv5l reached mAP of 91.60 % on multi-class objects detection in 15.50 ms per image.•The method reached total MA of 93.30 % with processing speed of 112.46 ms per image. Asynchrony of kiwifruit flowering time results in different flower phenological stages in canopy at the same time. Pollination quality of flowers is influenced by their phenological stages, while their distributions determine fruit distributions and influence kiwifruit quality and yield. Thus, it’s necessary to find suitable flowers to be pollinated based on flower phenology and its distribution. However, influences of flower phenology and flower distribution were not considered in most previous studies about robotic pollination of kiwifruit, where pollination of all open flowers was indiscriminate. Therefore, a method was proposed for multi-class detection of kiwifruit flower and its distribution identification in orchard, which was based on You Only Look Once version 5 large (YOLOv5l) and Euclidean distance. According to kiwifruit flower phenology, kiwifruit flowers were classified into 10 classes to find suitable flowers for pollination, while flower cluster and branch junction were divided into 4 classes for obtaining flower distributions. All classes were manually labeled by rectangular bounding boxes for training and testing. Considering high detection accuracy requirements with small model size, YOLOv5l was applied to do transfer learning for multi-class detection of kiwifruit flower. Then, pixels coordinate of multi-class objects and their corresponding Euclidean distances could be gained. Finally, flower distributions in canopy were obtained by matching method. Total mean Average Precision (mAP) was 91.60 % in YOLOv5l, while the mAP of multi-class flower (10 classes) was 93.23 %, which was 5.70 % higher than that of the other 4 classes. Matching accuracy (MA) of flowers matching flower clusters was up to 97.60 %. Moreover, MA of flower cluster matching branch junction (96.20 %) and total MA (93.30 %) increased by 1.20 % and 1.00 % based on improved matching method, respectively. Total processing speed of multi-class flower detection and its distribution identification was 112.46 ms per image including 15.50 ms for image detection by YOLOv