Content-Adaptive and Attention-Based Network for Hand Gesture Recognition

For hand gesture recognition, recurrent neural networks and 3D convolutional neural networks are the most commonly used methods for learning the spatial–temporal features of gestures. The calculation of the hidden state of the recurrent neural network at a specific time is determined by both input a...

Full description

Saved in:
Bibliographic Details
Published in:Applied sciences 2022-02, Vol.12 (4), p.2041
Main Authors: Cao, Zongjing, Li, Yan, Shin, Byeong-Seok
Format: Article
Language:English
Subjects:
Adaptive sampling
attention mechanism
Computer applications
content-adaptive
Convolution
Datasets
Gesture recognition
hand detection
Neural networks
Parallel processing
Recurrent neural networks
Spatial discrimination learning
Temporal variations
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:For hand gesture recognition, recurrent neural networks and 3D convolutional neural networks are the most commonly used methods for learning the spatial–temporal features of gestures. The calculation of the hidden state of the recurrent neural network at a specific time is determined by both input at the current time and the output of the hidden state at the previous time, therefore limiting its parallel computation. The large number of weight parameters that need to be optimized leads to high computational costs associated with 3D convolution-based methods. We introduced a transformer-based network for hand gesture recognition, which is a completely self-attentional architecture without any convolution or recurrent layers. The framework classifies hand gestures by focusing on the sequence information of the whole gesture video. In addition, we introduced an adaptive sampling strategy based on the video content to reduce the input of gesture-free frames to the model, thus reducing computational consumption. The proposed network achieved 83.2% and 93.8% recognition accuracy on two publicly available benchmark datasets, NVGesture and EgoGesture datasets, respectively. The results of extensive comparison experiments show that our proposed approach outperforms the existing state-of-the-art gesture recognition systems.
ISSN:2076-3417
2076-3417
DOI:10.3390/app12042041