A 4096-Neuron 1M-Synapse 3.8-pJ/SOP Spiking Neural Network With On-Chip STDP Learning and Sparse Weights in 10-nm FinFET CMOS

• Published in: IEEE journal of solid-state circuits 2019-04, Vol.54 (4), p.992-1002
• Main Authors: Chen, Gregory K., Kumar, Raghavan, Sumbul, H. Ekin, Knag, Phil C., Krishnamurthy, Ram K.
• Format: Article
• Language: English
• Subjects: Back propagation; Biological neural networks; Classification; CMOS; Digital electronics; Digits; Flow control; History; Image reconstruction; Learning; Multicasting; Multilayer perceptrons; Near-threshold voltage circuits; Neural networks; neuromorphic computing; Sparsity; spike-timing-dependent plasticity (STDP); Spiking; spiking neural networks (SNNs); Synapses; System-on-chip; Training; Weight reduction; weight sparsity
• ISSN: 0018-9200; 1558-173X
• DOI: 10.1109/JSSC.2018.2884901

A reconfigurable 4096-neuron, 1M-synapse chip in 10-nm FinFET CMOS is developed to accelerate inference and learning for many classes of spiking neural networks (SNNs). The SNN features digital circuits for leaky integrate and fire neuron models, on-chip spike-timing-dependent plasticity (STDP) learning, and high-fan-out multicast spike communication. Structured fine-grained weight sparsity reduces synapse memory by up to 16 \times with less than 2% overhead for storing connections. Approximate computing co-optimizes the dropping flow control and benefits from algorithmic noise to process spatiotemporal spike patterns with up to 9.4 \times lower energy. The SNN achieves a peak throughput of 25.2 GSOP/s at 0.9 V, peak energy efficiency of 3.8 pJ/SOP at 525 mV, and 2.3- \mu \text{W} /neuron operation at 450 mV. On-chip unsupervised STDP trains a spiking restricted Boltzmann machine to de-noise Modified National Institute of Standards and Technology (MNIST) digits and to reconstruct natural scene images with RMSE of 0.036. Near-threshold operation, in conjunction with temporal and spatial sparsity, reduces energy by 17.4\times to 1.0- \mu \text{J} /classification in a 236 \times 20 feed-forward network that is trained to classify MNIST digits using supervised STDP. A binary-activation multilayer perceptron with 50% sparse weights is trained offline with error backpropagation to classify MNIST digits with 97.9% accuracy at 1.7- \mu \text{J} /classification.