Dynamic hand gesture recognition using motion pattern and shape descriptors

The key problems of dynamic hand gesture recognition are large intra-class (gesture types, without considering hand configuration) spatial-temporal variability and similar inter-class (gesture types, only considering hand configuration) motion pattern. Firstly, for intra-class spatial-temporal varia...

Full description

Saved in:
Bibliographic Details
Published in:Multimedia tools and applications 2019-04, Vol.78 (8), p.10649-10672
Main Authors: Xing, Meng, Hu, Jing, Feng, Zhiyong, Su, Yong, Peng, Weilong, Zheng, Jinqing
Format: Article
Language:English
Subjects:
Cameras
Computer Communication Networks
Computer Science
Configurations
Data Structures and Information Theory
Forecasting
Gesture recognition
Histograms
Multimedia
Multimedia Information Systems
Pattern recognition
Shape recognition
Spacetime
Special Purpose and Application-Based Systems
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The key problems of dynamic hand gesture recognition are large intra-class (gesture types, without considering hand configuration) spatial-temporal variability and similar inter-class (gesture types, only considering hand configuration) motion pattern. Firstly, for intra-class spatial-temporal variability, the key is to reduce the spatial-temporal variability. Due to the average operation can improve the robustness very well, we propose a motion pattern descriptor, Time-Wise Histograms of Oriented Gradients (TWHOG), which extracts the average spatial-temporal information in the space-time domain from three orthogonal projection views (XY, YT, XT). Secondly, for similar inter-class motion pattern, accurate representation of hand configuration is especially important. Therefore, the difference in detail needs to be fully captured, and the shape descriptor can amplify subtle differences. Specifically, we introduce Depth Motion Maps-based Histograms of Oriented Gradients (DMM-HOG) to capture subtle differences in hand configurations between different types of gestures with similar motion patterns. Finally, we concatenate TWHOG and DMM-HOG to form the final feature vector Time-Shape Histograms of Oriented Gradients (TSHOG) and verify the effectiveness of the connection from quantitative and qualitative perspective. Comparison study with the state-of-the-art approaches are conducted on two challenge depth gesture datasets (MSRGesture3D, SKIG). The experiment result shows that TSHOG can achieve satisfactory performance while keeping a relative simple model with lower complexity as well as higher generality.
ISSN:1380-7501
1573-7721
DOI:10.1007/s11042-018-6553-9