Tracking individual honeybees among wildflower clusters with computer vision-facilitated pollinator monitoring

• Published in: PloS one 2021-02, Vol.16 (2), p.e0239504-e0239504
• Main Authors: Ratnayake, Malika Nisal, Dyer, Adrian G, Dorin, Alan
• Format: Article
• Language: English
• Subjects: Algorithms; Analysis; Animal behavior; Bees; Behavior; Biology and Life Sciences; Computer and Information Sciences; Computer vision; Deep learning; Honeybee; Information technology; Insects; Laboratories; Machine vision; Methods; Monitoring; Neural networks; Outdoors; Physical Sciences; Plant reproduction; Pollination; Pollination by insects; Research and Analysis Methods; Social Sciences; Tracking; Tracking and trailing
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0239504

Monitoring animals in their natural habitat is essential for advancement of animal behavioural studies, especially in pollination studies. Non-invasive techniques are preferred for these purposes as they reduce opportunities for research apparatus to interfere with behaviour. One potentially valuable approach is image-based tracking. However, the complexity of tracking unmarked wild animals using video is challenging in uncontrolled outdoor environments. Out-of-the-box algorithms currently present several problems in this context that can compromise accuracy, especially in cases of occlusion in a 3D environment. To address the issue, we present a novel hybrid detection and tracking algorithm to monitor unmarked insects outdoors. Our software can detect an insect, identify when a tracked insect becomes occluded from view and when it re-emerges, determine when an insect exits the camera field of view, and our software assembles a series of insect locations into a coherent trajectory. The insect detecting component of the software uses background subtraction and deep learning-based detection together to accurately and efficiently locate the insect among a cluster of wildflowers. We applied our method to track honeybees foraging outdoors using a new dataset that includes complex background detail, wind-blown foliage, and insects moving into and out of occlusion beneath leaves and among three-dimensional plant structures. We evaluated our software against human observations and previous techniques. It tracked honeybees at a rate of 86.6% on our dataset, 43% higher than the computationally more expensive, standalone deep learning model YOLOv2. We illustrate the value of our approach to quantify fine-scale foraging of honeybees. The ability to track unmarked insect pollinators in this way will help researchers better understand pollination ecology. The increased efficiency of our hybrid approach paves the way for the application of deep learning-based techniques to animal tracking in real-time using low-powered devices suitable for continuous monitoring.