Semantic segmentation of in-field cotton bolls from the sky using deep convolutional neural networks

• Published in: Smart agricultural technology 2022-12, Vol.2, p.100045, Article 100045
• Main Authors: Singh, Naseeb, Tewari, V.K., Biswas, P.K., Dhruw, L.K., Pareek, C.M., Singh, H. Dayananda
• Format: Article
• Language: English
• Subjects: Convolutional neural network; cotton; Cotton boll recognition; data collection; farm labor; fiber quality; Image segmentation; Robotic harvesting; Semantic segmentation; Sky detection
• ISSN: 2772-3755; 2772-3755
• DOI: 10.1016/j.atech.2022.100045

•Performed sematic segmentation of cotton bolls and sky.•Fully convolutional neural networks were used as encoders.•Models successfully discriminate between the pixels of cotton bolls and sky.•Achieved a maximum IoU of 84.5% and 80.67% for cotton bolls and sky, respectively. Manually picking of cotton bolls is a tedious, costly, and labor-intensive task, while harvesting using machines results in higher harvesting losses. By keeping selective picking in mind to maintain fiber quality, minimize harvesting losses, and tackle the shortage of farm labor in near future, cotton harvesting robots seem to be a better alternative in coming years in both developing and developed countries. For the cotton harvesting robot, cotton boll recognition with minimum errors is a foremost and challenging task. While recognizing cotton bolls, false-positive errors occur due to sky interference. In present study, convolutional neural networks were used to segment and discriminate the cotton bolls pixels from sky pixels. For that, three fully convolutional neural networks namely VGG16, InceptionV3, and ResNet34 were used as encoders and trained. These trained neural networks models were evaluated using the intersection-over-union (IoU), F1-score, precision, and recall metrics. All proposed models were tested on a cotton-sky dataset and achieved an IoU score of above 81% and 80% for cotton bolls and sky, respectively. InceptionV3 model outperforms with an IoU score of 84.5% and 80.67% for cotton bolls and sky, respectively with a segmentation time of 1.07 s. For the cotton dataset, proposed models achieved an IoU score of above 90% for cotton bolls and the InceptionV3 model outperforms with an IoU score of 93.29%. It can be concluded that the InceptionV3 model segmented cotton bolls and sky with higher accuracy, and low error rates and, hence can be deployed to cotton harvesting robots effectively.