Human In-Hand Motion Recognition Based on Multi-Modal Perception Information Fusion

A human in-hand motion (HIM) recognition system based on multi-modal perception information fusion is proposed in this paper, which can observe the state information between the object and the hand by using customized ten kinds of HIM manipulation in order to recognize the complex HIMs. First, combi...

Full description

Saved in:
Bibliographic Details
Published in:IEEE sensors journal 2022-04, Vol.22 (7), p.6793-6805
Main Authors: Xue, Yaxu, Yu, Yadong, Yin, Kaiyang, Li, Pengfei, Xie, Shuangxi, Ju, Zhaojie
Format: Article
Language:English
Subjects:
Character recognition
Contact force
Data acquisition
Data analysis
Data integration
Empirical analysis
empirical mode decomposition
Feature extraction
Force
Gesture recognition
human in-hand motion
Human motion
Liapunov exponents
Machine learning
maximum Lyapunov exponent
Modal data
Motion perception
Multi-modal information
random forest
Recognition
Segmentation
Sensor phenomena and characterization
Sensors
Wavelet transforms
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:A human in-hand motion (HIM) recognition system based on multi-modal perception information fusion is proposed in this paper, which can observe the state information between the object and the hand by using customized ten kinds of HIM manipulation in order to recognize the complex HIMs. First, combined with the characteristics of HIM capture, ten kinds of HIM sets are designed, and finger trajectory, contact force and electromyographic signal data are acquired synchronously through the multi-modal data acquisition platform; second, motion segmentation is realized through the threshold segmentation method, the multi-modal signal preprocessing is realized by Empirical Mode Decomposition (EMD), and multi-modal signal feature extraction is realized by Maximum Lyapunov Exponent (MLE); then, a detailed non-linear data analysis is carried out. A detailed analysis and discussion are presented from the results of the Random Forest (RF) recognizing HIMs, the comparison results of motion recognition rates of different subjects, the comparison results of motion recognition rates of different perceptrons, and the comparison results of the motion recognition rates of different machine learning methods. The experimental results show that the multi-modal perception information based HIM recognition system proposed in this paper can effectively recognize ten different HIMs, with an accuracy rate of 93.72%.
ISSN:1530-437X
1558-1748
DOI:10.1109/JSEN.2022.3148992