Training environmental sound classification models for real-world deployment in edge devices

• Published in: Discover applied sciences 2024-03, Vol.6 (4), p.166, Article 166
• Main Authors: Goulão, Manuel, Bandeira, Lourenço, Martins, Bruno, L. Oliveira, Arlindo
• Format: Article
• Language: English
• Subjects: Accuracy; Background noise; Bicycles; Classification; Data collection; Datasets; Deep learning; Distillation; Energy consumption; Environment models; Information processing; Knowledge; Machine learning; Neural networks; Ontology; Sensors; Sirens; Traffic volume
• ISSN: 3004-9261; 2523-3963; 3004-9261; 2523-3971
• DOI: 10.1007/s42452-024-05803-7

The interest in smart city technologies has grown in recent years, and a major challenge is to develop methods that can extract useful information from data collected by sensors in the city. One possible scenario is the use of sound sensors to detect passing vehicles, sirens, and other sounds on the streets. However, classifying sounds in a street environment is a complex task due to various factors that can affect sound quality, such as weather, traffic volume, and microphone quality. This paper presents a deep learning model for multi-label sound classification that can be deployed in the real world on edge devices. We describe two key components, namely data collection and preparation, and the methodology to train the model including a pre-train using knowledge distillation. We benchmark our models on the ESC-50 dataset and show an accuracy of 85.4%, comparable to similar state-of-the-art models requiring significantly more computational resources. We also evaluated the model using data collected in the real world by early prototypes of luminaires integrating edge devices, with results showing that the approach works well for most vehicles but has significant limitations for the classes “person” and “bicycle”. Given the difference between the benchmarking and the real-world results, we claim that the quality and quantity of public and private data for this type of task is the main limitation. Finally, all results show great benefits in pretraining the model using knowledge distillation.