Neural Network Training Acceleration With RRAM-Based Hybrid Synapses

• Published in: Frontiers in neuroscience 2021-06, Vol.15, p.690418-690418
• Main Authors: Choi, Wooseok, Kwak, Myonghoon, Kim, Seyoung, Hwang, Hyunsang
• Format: Article
• Language: English
• Subjects: Accuracy; Artificial intelligence; Conductance; crossbar array; hardware neural networks; hybrid synapse; Learning; Neural networks; Neurons; Neuroscience; Number systems; online training; resistive memory
• ISSN: 1662-453X; 1662-4548; 1662-453X
• DOI: 10.3389/fnins.2021.690418

Hardware neural network (HNN) based on analog synapse array excels in accelerating parallel computations. To implement an energy-efficient HNN with high accuracy, high-precision synaptic devices and fully-parallel array operations are essential. However, existing resistive memory (RRAM) devices can represent only a finite number of conductance states. Recently, there have been attempts to compensate device nonidealities using multiple devices per weight. While there is a benefit, it is difficult to apply the existing parallel updating scheme to the synaptic units, which significantly increases updating process’s cost in terms of computation speed, energy, and complexity. Here, we propose an RRAM-based hybrid synaptic unit consisting of a “big” synapse and a “small” synapse, and a related training method. Unlike previous attempts, array-wise fully-parallel learning is possible with our proposed architecture with a simple array selection logic. To experimentally verify the hybrid synapse, we exploit Mo/TiO x RRAM, which shows promising synaptic properties and areal dependency of conductance precision. By realizing the intrinsic gain via proportionally scaled device area, we show that the big and small synapse can be implemented at the device-level without modifications to the operational scheme. Through neural network simulations, we confirm that RRAM-based hybrid synapse with the proposed learning method achieves maximum accuracy of 97 %, comparable to floating-point implementation (97.92%) of the software even with only 50 conductance states in each device. Our results promise training efficiency and inference accuracy by using existing RRAM devices.