Using a two-stage convolutional neural network to rapidly identify tiny herbivorous beetles in the field

• Published in: Ecological informatics 2021-12, Vol.66, p.101466, Article 101466
• Main Authors: Takimoto, Hironori, Sato, Yasuhiro, Nagano, Atsushi J., Shimizu, Kentaro K., Kanagawa, Akihiro
• Format: Article
• Language: English
• Subjects: Deep learning; Entomology; Fine-grained image classification; Herbivory; Small object detection
• ISSN: 1574-9541
• DOI: 10.1016/j.ecoinf.2021.101466

•We proposed a two-stage convolutional neural network (CNN) method in this study.•In this method, YOLOv4 and EfficientNet were used for the detection of small insects.•The two-stage method was applied to a field video of 2-mm long beetles.•Our method classified the beetles and backgrounds more precisely than did YOLO alone. Recently, deep convolutional neural networks (CNN) have been adopted to help non-experts identify insect species from field images. However, the application of these methods on the rapid identification of tiny congeneric species moving across heterogeneous background remains difficult. To improve rapid and automatic identification in the field, we customized an existing CNN-based method for a field video involving two Phyllotreta beetles. We first performed data augmentation using transformations, syntheses, and random erasing of the original images. We then proposed a two-stage method for the detection and identification of small insects based on CNN, where YOLOv4 and EfficientNet were used as a detector and a classifier, respectively. Evaluation of the model revealed that one-step object detection by YOLOv4 alone was not precise (Precision=0.55) when classifying two species of flea beetles and background objects. In contrast, the two-step CNNs improved the precision (Precision=0.89) with moderate accuracy (F-measure=0.55) and acceptable speed (ca. 5 frames per second for full HD images) of detection and identification of insect species in the field. Although real-time identification of tiny insects remains a challenge in the field, our method aids in improving small object detection on a heterogeneous background.