BinaryDenseNet: Developing an Architecture for Binary Neural Networks

Binary Neural Networks (BNNs) show promising progress in reducing computational and memory costs, but suffer from substantial accuracy degradation compared to their real-valued counterparts on large-scale datasets, e.g., ImageNet. In this work we study existing BNN architectures and revisit the comm...

Full description

Saved in:
Bibliographic Details
Main Authors: Bethge, Joseph, Yang, Haojin, Bornstein, Marvin, Meinel, Christoph
Format: Conference Proceeding
Language:English
Subjects:
Acceleration
architecture
binary densenet
binary neural network
Computer architecture
Convolution
Degradation
Neural networks
Quantization (signal)
Training
Citations: Items that cite this one
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Binary Neural Networks (BNNs) show promising progress in reducing computational and memory costs, but suffer from substantial accuracy degradation compared to their real-valued counterparts on large-scale datasets, e.g., ImageNet. In this work we study existing BNN architectures and revisit the commonly used technique to include scaling factors. We suggest several architectural design principles for BNNs, based on our studies on architectures. Guided by our principles we develop a novel BNN architecture BinaryDenseNet, which is the first architecture specifically created for BNNs to the best of our knowledge. In our experiments, BinaryDenseNet achieves 18.6% and 7.6% relative improvement over the well-known XNOR-Network and the current state-of-the-art Bi-Real Net in terms of top-1 accuracy on ImageNet, respectively. Further, we show the competitiveness of our BinaryDenseNet regarding memory requirements and computational complexity.
ISSN:2473-9944
DOI:10.1109/ICCVW.2019.00244