Transforming unmanned pineapple picking with spatio-temporal convolutional neural networks

• Published in: Computers and electronics in agriculture 2023-11, Vol.214, p.108298, Article 108298
• Main Authors: Meng, Fan, Li, Jinhui, Zhang, Yunqi, Qi, Shaojun, Tang, Yunchao
• Format: Article
• Language: English
• Subjects: Computer vision; Deep learning; Pineapple detection; Shifted window Transformer; Spatio-temporal convolutional neural network
• ISSN: 0168-1699; 1872-7107
• DOI: 10.1016/j.compag.2023.108298

Automated pineapple harvesting has emerged as a prominent prospective development within the agricultural domain. Nevertheless, the intricate growth conditions that pineapples encounter in the field, such as inadequate light, overexposure, obstructions caused by fruit leaves, or the overlapping of fruits, pose substantial challenges to the accuracy and robustness of traditional real-time detection algorithms. In recent times, Transformer models, when applied to computer vision, have exhibited commendable performance, underscoring their potential for target detection in smart agricultural applications. In this report, we propose a spatio-temporal convolutional neural network model that leverages the shifted window Transformer fusion region convolutional neural network model for the purpose of detecting pineapple fruits. Our study includes a comparative analysis of these results and those obtained through the utilization of conventional models. Additionally, we investigate the influence of various aspects of data preparation, including image resolution, object size, and object complexity, on the ultimate pineapple detection outcomes. Experimental findings elucidate that, in the case of detecting a single-category target like a pineapple, the employment of 2000 annotated supervised data points yields the optimal detection accuracy. Further augmenting the size of the training dataset does not yield any significant improvement in detection accuracy. Furthermore, images of pineapples captured from greater distances encompass smaller targets and an increased number of pineapple instances, rendering them more intricate and challenging to accurately detect. In summary, our study employs the spatio-temporal convolutional neural network model to attain pineapple detection with an impressive accuracy rate of 92.54% and an average inference time of 0.163 s, thus affirming the efficacy of our developed model in achieving superior detection results.