EchoWrite-SNN: Acoustic Based Air-Written Shape Recognition Using Spiking Neural Networks

• Main Authors: George, Arun M, Gigie, Andrew, Kumar, A Anil, Dey, Sounak, Pal, Arpan, Aditi, K
• Format: Conference Proceeding
• Language: English
• Subjects: air-writing; finger tracking; Neural networks; neuromorphic computing; Neuromorphic engineering; Performance evaluation; Shape; SONAR; Sonar applications; Spiking neural network; Tracking; Ultrasonic imaging
• ISSN: 2158-1525
• DOI: 10.1109/ISCAS48785.2022.9937951

In this paper, we propose EchoWrite-SNN, a robust edge compatible air-writing recognition system (used in applications such as AR/VR, HRI etc.) based on principles of SONAR and neuromorphic computing. The bare finger movements in air are captured by a pair of commonly available speaker-microphone pair. A new tracking algorithm based on windowed difference cross-correlation and ESPRIT is employed which shows better tracking accuracy compared to state-of-the-art methods with a median tracking error of only 3.31mm. To classify these air-written shapes, a 5-layer CNN is trained and then converted to a Spiking Neural Network (SNN) using ANN-to-SNN conversion technique to reap the benefits of low power neuromorphic computing on edge. Experimental results show that the converted SNN achieves 92% accuracy (a mere 3% less than the CNN) while showing 4.4 × reduction in number of operations compared to CNN resulting in further energy benefit when run on actual neuromorphic computation platforms.