HardCoRe-NAS: Hard Constrained diffeRentiable Neural Architecture Search

Realistic use of neural networks often requires adhering to multiple constraints on latency, energy and memory among others. A popular approach to find fitting networks is through constrained Neural Architecture Search (NAS), however, previous methods enforce the constraint only softly. Therefore, t...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2021-02
Main Authors: Nayman, Niv, Aflalo, Yonathan, Noy, Asaf, Zelnik-Manor, Lihi
Format: Article
Language:English
Subjects:
Computer architecture
Constraints
Network latency
Neural networks
Search methods
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Realistic use of neural networks often requires adhering to multiple constraints on latency, energy and memory among others. A popular approach to find fitting networks is through constrained Neural Architecture Search (NAS), however, previous methods enforce the constraint only softly. Therefore, the resulting networks do not exactly adhere to the resource constraint and their accuracy is harmed. In this work we resolve this by introducing Hard Constrained diffeRentiable NAS (HardCoRe-NAS), that is based on an accurate formulation of the expected resource requirement and a scalable search method that satisfies the hard constraint throughout the search. Our experiments show that HardCoRe-NAS generates state-of-the-art architectures, surpassing other NAS methods, while strictly satisfying the hard resource constraints without any tuning required.
ISSN:2331-8422