Neural sampling machine with stochastic synapse allows brain-like learning and inference

• Published in: Nature communications 2022-05, Vol.13 (1), p.2571-2571, Article 2571
• Main Authors: Dutta, Sourav, Detorakis, Georgios, Khanna, Abhishek, Grisafe, Benjamin, Neftci, Emre, Datta, Suman
• Format: Article
• Language: English
• Subjects: 639/166/987; 639/301/1005/1007; 639/766/1130; 639/925/357; 639/925/927/1007; Artificial Intelligence; Bayes Theorem; Bayesian analysis; Brain; Confidence intervals; Datasets; Decision making; Distance learning; Entropy; Ferroelectric materials; Ferroelectricity; Field effect transistors; Hardware; Humanities and Social Sciences; Image classification; Learning; Machine learning; Mathematical models; multidisciplinary; Nervous system; Neural networks; Neural Networks, Computer; Noise; Probabilistic models; Probability theory; Sampling; Science; Science (multidisciplinary); Semiconductor devices; Statistical analysis; Statistical inference; Stochasticity; Synapses; Transistors; Uncertainty
• ISSN: 2041-1723; 2041-1723
• DOI: 10.1038/s41467-022-30305-8

Many real-world mission-critical applications require continual online learning from noisy data and real-time decision making with a defined confidence level. Brain-inspired probabilistic models of neural network can explicitly handle the uncertainty in data and allow adaptive learning on the fly. However, their implementation in a compact, low-power hardware remains a challenge. In this work, we introduce a novel hardware fabric that can implement a new class of stochastic neural network called Neural Sampling Machine (NSM) by exploiting the stochasticity in the synaptic connections for approximate Bayesian inference. We experimentally demonstrate an in silico hybrid stochastic synapse by pairing a ferroelectric field-effect transistor (FeFET)-based analog weight cell with a two-terminal stochastic selector element. We show that the stochastic switching characteristic of the selector between the insulator and the metallic states resembles the multiplicative synaptic noise of the NSM. We perform network-level simulations to highlight the salient features offered by the stochastic NSM such as performing autonomous weight normalization for continual online learning and Bayesian inferencing. We show that the stochastic NSM can not only perform highly accurate image classification with 98.25% accuracy on standard MNIST dataset, but also estimate the uncertainty in prediction (measured in terms of the entropy of prediction) when the digits of the MNIST dataset are rotated. Building such a probabilistic hardware platform that can support neuroscience inspired models can enhance the learning and inference capability of the current artificial intelligence (AI). Neural sampling machines make use of noise to perform learning. Here, Dutta et al. present a hybrid stochastic synapse composed out of a ferroelectric transistor combined with a stochastic selector exhibiting multiplicative synaptic noise required for implementing a neural sample machine.