Robust Binary Neural Network against Noisy Analog Computation

Computing in memory (CIM) technology has shown promising results in reducing the energy consumption of a battery-powered device. On the other hand, to reduce MAC operations, Binary neural networks (BNN) show the potential to catch up with a full-precision model. This paper proposes a robust BNN mode...

Full description

Saved in:
Bibliographic Details
Main Authors: Lee, Zong-Han, Tsai, Fu-Cheng, Chang, Shih-Chieh
Format: Conference Proceeding
Language:English
Subjects:
Analog AI
Computational modeling
Deep neural networks
Microprocessors
Neural networks
Noise tolerance
Power demand
Robustness
Standardization
Training
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Computing in memory (CIM) technology has shown promising results in reducing the energy consumption of a battery-powered device. On the other hand, to reduce MAC operations, Binary neural networks (BNN) show the potential to catch up with a full-precision model. This paper proposes a robust BNN model applied to the CIM framework, which can tolerate analog noises. These analog noises caused by various variations, such as process variation, can lead to low inference accuracy. We first observe that the traditional batch normalization can cause a BNN model to be susceptible to analog noise. We then propose a new approach to replace the batch normalization while maintaining the advantages. Secondly, in BNN, since noises can be removed when inputs are zeros during the multiplication and accumulation (MAC) operation, we also propose novel methods to increase the number of zeros in a convolution output. We apply our new BNN model in the keyword spotting application. Our results are very exciting.
ISSN:1558-1101
DOI:10.23919/DATE54114.2022.9774565