Extensive data engineering to the rescue: building a multi-species katydid detector from unbalanced, atypical training datasets

• Published in: Philosophical transactions of the Royal Society of London. Series B. Biological sciences 2024-06, Vol.379 (1904), p.20230444-20230444
• Main Authors: Madhusudhana, Shyam, Klinck, Holger, Symes, Laurel B
• Format: Article
• Language: English
• Subjects: Acoustics; Animals; Deep Learning; Panama; Species Specificity; Vocalization, Animal
• ISSN: 0962-8436; 1471-2970
• DOI: 10.1098/rstb.2023.0444

Passive acoustic monitoring (PAM) is a powerful tool for studying ecosystems. However, its effective application in tropical environments, particularly for insects, poses distinct challenges. Neotropical katydids produce complex species-specific calls, spanning mere milliseconds to seconds and spread across broad audible and ultrasonic frequencies. However, subtle differences in inter-pulse intervals or central frequencies are often the only discriminatory traits. These extremities, coupled with low source levels and susceptibility to masking by ambient noise, challenge species identification in PAM recordings. This study aimed to develop a deep learning-based solution to automate the recognition of 31 katydid species of interest in a biodiverse Panamanian forest with over 80 katydid species. Besides the innate challenges, our efforts were also encumbered by a limited and imbalanced initial training dataset comprising domain-mismatched recordings. To overcome these, we applied rigorous data engineering, improving input variance through controlled playback re-recordings and by employing physics-based data augmentation techniques, and tuning signal-processing, model and training parameters to produce a custom well-fit solution. Methods developed here are incorporated into Koogu, an open-source Python-based toolbox for developing deep learning-based bioacoustic analysis solutions. The parametric implementations offer a valuable resource, enhancing the capabilities of PAM for studying insects in tropical ecosystems. This article is part of the theme issue 'Towards a toolkit for global insect biodiversity monitoring'.