TinyEmergencyNet: a hardware-friendly ultra-lightweight deep learning model for aerial scene image classification

• Published in: Journal of real-time image processing 2024-04, Vol.21 (2), p.51, Article 51
• Main Authors: Mogaka, Obed M., Zewail, Rami, Inoue, Koji, Sayed, Mohammed S.
• Format: Article
• Language: English
• Subjects: Artificial neural networks; Classification; Co-design; Computer Graphics; Computer Science; Deep learning; Design optimization; Emergency response; Floating point arithmetic; Hardware; Image classification; Image processing; Image Processing and Computer Vision; Machine learning; Multimedia Information Systems; Neural networks; Optimization techniques; Pattern Recognition; Pruning; Remote sensing; Signal,Image and Speech Processing; Situational awareness; Software; Sparsity; Unmanned aerial vehicles
• ISSN: 1861-8200; 1861-8219
• DOI: 10.1007/s11554-024-01430-y

In the context of emergency response applications, real-time situational awareness is vital. Unmanned aerial vehicles (UAVs) with imagers have emerged as crucial tools for providing timely information in such scenarios. Convolutional neural networks (CNN) are effective in image processing. However, the deployment of CNN models in UAVs faces significant challenges. The CNN models involve large number of parameters and energy-costly floating-point computations beyond the memory and power available on-board the UAVs. To address these challenges, we propose a co-design optimization approach for deploying the EmergencyNet CNN model on resource-constrained UAVs. Our strategy includes channel-wise pruning to reduce the size and optimize the network architecture. Additionally, we apply additive powers-of-two (APoT) quantization to further compress the model and enhance computational efficiency. Using channel-wise network pruning we derive TinyEmergencyNet that is only 155KB in memory size and 50% smaller than EmergencyNet. This proposed approach is evaluated on Aerial Image Disaster Event Recognition (AIDER) dataset. We have achieved an F1-score of 93.6% with 4-bit APoT quantization that closely approaches the full precision (32-bit) accuracy of 94%. Furthermore, hardware-friendly bit-shifting operations as a result of APoT quantization present an added advantage in hardware accelerator implementations. This work pioneers the joint application of channel-wise pruning and non-uniform APoT quantization on EmergencyNet, presenting a suitable solution tailored for UAV-based emergency response applications.