Equilibrium Propagation for Memristor-Based Recurrent Neural Networks

• Published in: Frontiers in neuroscience 2020-03, Vol.14, p.240-240
• Main Authors: Zoppo, Gianluca, Marrone, Francesco, Corinto, Fernando
• Format: Article
• Language: English
• Subjects: Algorithms; artificial neural network; associative memory; Back propagation; biologically plausible learning rule; Deep learning; Efficiency; Equilibrium; Integrated circuits; Investigations; Learning; Memory; memristor; Neural networks; Neurobiology; neuromorphic computing; Neurons; Neuroscience; Nonlinear systems; Propagation; recurrent neural network; Synaptic strength
• ISSN: 1662-4548; 1662-453X; 1662-453X
• DOI: 10.3389/fnins.2020.00240

Among the recent innovative technologies, memristor (memory-resistor) has attracted researchers attention as a fundamental computation element. It has been experimentally shown that memristive elements can emulate synaptic dynamics and are even capable of supporting spike timing dependent plasticity (STDP), an important adaptation rule that is gaining particular interest because of its simplicity and biological plausibility. The overall goal of this work is to provide a novel (theoretical) analog computing platform based on memristor devices and recurrent neural networks that exploits the memristor device physics to implement two variations of the backpropagation algorithm: recurrent backpropagation and equilibrium propagation. In the first learning technique, the use of memristor-based synaptic weights permits to propagate the error signals in the network by means of the nonlinear dynamics via an analog side network. This makes the processing non-digital and different from the current procedures. However, the necessity of a side analog network for the propagation of error derivatives makes this technique still highly biologically implausible. In order to solve this limitation, it is therefore proposed an alternative solution to the use of a side network by introducing a learning technique used for energy-based models: equilibrium propagation. Experimental results show that both approaches significantly outperform conventional architectures used for pattern reconstruction. Furthermore, due to the high suitability for VLSI implementation of the equilibrium propagation learning rule, additional results on the classification of the MNIST dataset are here reported.