Tomato flower pollination features recognition based on binocular gray value-deformation coupled template matching

• Published in: Computers and electronics in agriculture 2023-11, Vol.214, p.108345, Article 108345
• Main Authors: Liu, Siyao, Zhang, Xuemin, Wang, Xiaoyan, Hou, Xiuning, Chen, Xiangan, Xu, Jing
• Format: Article
• Language: English
• Subjects: agriculture; binocular vision; computer vision; Deep learning; deformation; electronics; Flower recognition; flowers; greenhouses; light intensity; lighting; pollination; Pollination features; Stereo vision; Template matching; tomatoes; vision
• ISSN: 0168-1699; 1872-7107
• DOI: 10.1016/j.compag.2023.108345

•Improved MC-AlexNet structure is established for opening state recognition.•Binocular grayvalue deformation flower template matching method is designed.•Another 3D position and pose recognition method is established. It is necessary to recognize the tomato pollination features for the designing demand of intelligent and precise tomato supplementary pollination equipment. Mentioned pollination features include flower opening state and the three-dimensional position and pose of flower anther. Tomato flower pollination features recognition method is designed in this paper based on the deep learning full opened flower recognition model and binocular template matching three-dimensional information recognition method. First of all, a binocular stereo vision system is built to acquire the tomato flower images in greenhouse with natural lighting. Which can help vision system avoid the impact of inconsistent light intensity on image recognition. The acquired images were equalized with three groups parameters to labeled images. And the improved MC-AlexNet deep learning model is established to recognize the full opened tomato flowers in the left image of image pair acquired with binocular vision system. Then, the template is created with the full opened flower recognition result of deep learning model in left image based on gray value and deformation template matching method. And the template matching is established to recognize the corresponding full opened flowers in the right image of image pair. Finally, with the template matching result, the anther segmentation is conducted to calculate three-dimensional position and pose of anther. The experimental results show that the accuracy of full opened tomato flowers recognition model is 96.23%. The average position recognition deviation of anther is 5.94 mm. And the average anther pose recognition deviation in three plane is 6.24° for the images that anthers can be observed. And the average time consuming is about 143.18 ms per image pair. It can be concluded that the method of binocular template matching established in this paper can fulfill the demand of supplementary pollination equipment design, and the research result lays a foundation for designing and improvement of tomato supplementary pollination equipment in greenhouse.