Conversion of Siamese networks to spiking neural networks for energy-efficient object tracking

Recently, spiking neural networks (SNNs), the third generation of neural networks, have shown remarkable capabilities of energy-efficient computing, which is a promising alternative for artificial neural networks (ANNs) with high energy consumption. SNNs have reached competitive results compared to...

Full description

Saved in:
Bibliographic Details
Published in:Neural computing & applications 2022-06, Vol.34 (12), p.9967-9982
Main Authors: Luo, Yihao, Shen, Haibo, Cao, Xiang, Wang, Tianjiang, Feng, Qi, Tan, Zehan
Format: Article
Language:English
Subjects:
Artificial Intelligence
Artificial neural networks
Computational Biology/Bioinformatics
Computational Science and Engineering
Computer Science
Data Mining and Knowledge Discovery
Datasets
Energy consumption
Feature maps
Image classification
Image Processing and Computer Vision
Network latency
Neural networks
Optical tracking
Original Article
Probability and Statistics in Computer Science
Spiking
Task complexity
Temporal distribution
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Recently, spiking neural networks (SNNs), the third generation of neural networks, have shown remarkable capabilities of energy-efficient computing, which is a promising alternative for artificial neural networks (ANNs) with high energy consumption. SNNs have reached competitive results compared to ANNs in relatively simple tasks and small datasets such as image classification and MNIST/CIFAR, while few studies on more challenging vision tasks on complex datasets. In this paper, we focus on extending deep SNNs to object tracking, a more advanced vision task with embedded applications and energy-saving requirements. We present a spike-based Siamese network called SiamSNN, which is converted from fully convolutional Siamese networks. Specifically, we propose a spiking correlation layer to evaluate the similarity between two spiking feature maps, and introduce a novel two-status coding scheme to optimize the temporal distribution of output spike trains for further improvements. SiamSNN is the first deep SNN tracker that achieves short latency and low precision degradation on the visual object tracking benchmarks OTB-2013, OTB-2015, VOT-2016, VOT-2018, and GOT-10k. Moreover, SiamSNN achieves notably low energy consumption and real-time on Neuromorphic chip TrueNorth.
ISSN:0941-0643
1433-3058
DOI:10.1007/s00521-022-06984-1