A PointNet-Based Solution for 3D Hand Gesture Recognition

Gesture recognition is an intensively researched area for several reasons. One of the most important reasons is because of this technology's numerous application in various domains (e.g., robotics, games, medicine, automotive, etc.) Additionally, the introduction of three-dimensional (3D) image...

Full description

Saved in:
Bibliographic Details
Published in:Sensors (Basel, Switzerland) Switzerland), 2020-06, Vol.20 (11), p.3226
Main Authors: Mirsu, Radu, Simion, Georgiana, Caleanu, Catalin Daniel, Pop-Calimanu, Ioana Monica
Format: Article
Language:English
Subjects:
Algorithms
Architecture
Cameras
Classification
Clustering
Computer vision
Datasets
Deep learning
Domains
Forearm
Gesture recognition
Gestures
Hand
hand gesture recognition
Humans
Image acquisition
Methods
Neural networks
Neural Networks, Computer
Pattern Recognition, Automated
PointNet
Principal components analysis
Recognition
Recognition, Psychology
Sensors
time of flight sensors
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Gesture recognition is an intensively researched area for several reasons. One of the most important reasons is because of this technology's numerous application in various domains (e.g., robotics, games, medicine, automotive, etc.) Additionally, the introduction of three-dimensional (3D) image acquisition techniques (e.g., stereovision, projected-light, time-of-flight, etc.) overcomes the limitations of traditional two-dimensional (2D) approaches. Combined with the larger availability of 3D sensors (e.g., Microsoft Kinect, Intel RealSense, photonic mixer device (PMD), CamCube, etc.), recent interest in this domain has sparked. Moreover, in many computer vision tasks, the traditional statistic top approaches were outperformed by deep neural network-based solutions. In view of these considerations, we proposed a deep neural network solution by employing PointNet architecture for the problem of hand gesture recognition using depth data produced by a time of flight (ToF) sensor. We created a custom hand gesture dataset, then proposed a multistage hand segmentation by designing filtering, clustering, and finding the hand in the volume of interest and hand-forearm segmentation. For comparison purpose, two equivalent datasets were tested: a 3D point cloud dataset and a 2D image dataset, both obtained from the same stream. Besides the advantages of the 3D technology, the accuracy of the 3D method using PointNet is proven to outperform the 2D method in all circumstances, even the 2D method that employs a deep neural network.
ISSN:1424-8220
1424-8220
DOI:10.3390/s20113226