ColabNAS: Obtaining lightweight task-specific convolutional neural networks following Occam’s razor

• Published in: Future generation computer systems 2024-03, Vol.152, p.152-159
• Main Authors: Garavagno, Andrea Mattia, Leonardis, Daniele, Frisoli, Antonio
• Format: Article
• Language: English
• Subjects: Hardware-aware neural architecture search; Lightweight convolutional neural networks; TinyML; Visual wake words
• ISSN: 0167-739X; 1872-7115
• DOI: 10.1016/j.future.2023.11.003

The current trend of applying transfer learning from convolutional neural networks (CNNs) trained on large datasets can be an overkill when the target application is a custom and delimited problem, with enough data to train a network from scratch. On the other hand, the training of custom and lighter CNNs requires expertise, in the from-scratch case, and or high-end resources, as in the case of hardware-aware neural architecture search (HW NAS), limiting access to the technology by non-habitual NN developers. For this reason, we present ColabNAS, an affordable HW NAS technique for producing lightweight task-specific CNNs. Its novel derivative-free search strategy, inspired by Occam’s razor, allows to obtain state-of-the-art results on the Visual Wake Word dataset, a standard TinyML benchmark, in just 3.1 GPU hours using free online GPU services such as Google Colaboratory and Kaggle Kernel. •Hardware-aware neural architecture search algorithm for task-specific convolutional neural networks.•A novel low-cost derivative-free search strategy inspired by Occam’s razor.•State-of-the-art results on the Visual Wake Word dataset in just 3.1 GPU hours.•Able to be executed on free subscription online GPU services.•Convolutional Neural Networks for low-RAM microcontrollers (e.g. 40 kiB).