S-TLLR: STDP-inspired Temporal Local Learning Rule for Spiking Neural Networks

Spiking Neural Networks (SNNs) are biologically plausible models that have been identified as potentially apt for deploying energy-efficient intelligence at the edge, particularly for sequential learning tasks. However, training of SNNs poses significant challenges due to the necessity for precise t...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-10
Main Authors: Apolinario, Marco Paul E, Kaushik, Roy
Format: Article
Language:English
Subjects:
Algorithms
Back propagation networks
Cognitive tasks
Distance learning
Gesture recognition
Machine learning
Neural networks
Optical flow (image analysis)
Performance enhancement
Spiking
Training
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Spiking Neural Networks (SNNs) are biologically plausible models that have been identified as potentially apt for deploying energy-efficient intelligence at the edge, particularly for sequential learning tasks. However, training of SNNs poses significant challenges due to the necessity for precise temporal and spatial credit assignment. Back-propagation through time (BPTT) algorithm, whilst the most widely used method for addressing these issues, incurs high computational cost due to its temporal dependency. In this work, we propose S-TLLR, a novel three-factor temporal local learning rule inspired by the Spike-Timing Dependent Plasticity (STDP) mechanism, aimed at training deep SNNs on event-based learning tasks. Furthermore, S-TLLR is designed to have low memory and time complexities, which are independent of the number of time steps, rendering it suitable for online learning on low-power edge devices. To demonstrate the scalability of our proposed method, we have conducted extensive evaluations on event-based datasets spanning a wide range of applications, such as image and gesture recognition, audio classification, and optical flow estimation. In all the experiments, S-TLLR achieved high accuracy, comparable to BPTT, with a reduction in memory between \(5-50\times\) and multiply-accumulate (MAC) operations between \(1.3-6.6\times\).
ISSN:2331-8422