Transfer-Learning-Based Gesture and Pose Recognition System for Human-Robot Interaction: An Internet of Things Application

Human-machine interactions have become increasingly crucial in the current era of the Internet of Things (IoT). Mutual feedback is critical for adjusting machine operations to improve the efficiency of human-machine interactions. Imaging can be easily conducted in various contexts to acquire large v...

Full description

Saved in:
Bibliographic Details
Published in:IEEE internet of things journal 2024-11, Vol.11 (21), p.35376-35389
Main Authors: Kuo, Ping-Huan, Shen, Yu-Chi, Feng, Po-Hsun, Chiu, Yu-Jhih, Yau, Her-Terng
Format: Article
Language:English
Subjects:
Computational modeling
Data models
Fasteners
Hand tracking
Image recognition
Internet of Things
Machine learning
Manipulators
posture estimation
Recognition
Training
transfer learning
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Human-machine interactions have become increasingly crucial in the current era of the Internet of Things (IoT). Mutual feedback is critical for adjusting machine operations to improve the efficiency of human-machine interactions. Imaging can be easily conducted in various contexts to acquire large volumes of visual information, such as that regarding human gestures. In the present study, machine learning technology, which is the driving technology for intelligent processing in IoT systems, was adopted to develop a system for identifying and classifying six hand gestures and five body poses. Gesture and pose data were collected and analyzed using multiple algorithms to construct classification models for pose recognition. Data for one individual were used to train base gesture and pose recognition models, and transfer learning was then performed to adapt these base models to the gesture and pose data of other individuals. The adapted models achieved satisfactory recognition accuracy. The developed gesture and pose recognition models were tested by employing them to control a robotic arm and an automated guided vehicle, respectively. All models achieved accuracy rates of >97%, thereby confirming the effectiveness of the proposed machine-learning-based method for gesture and pose recognition.
ISSN:2327-4662
2327-4662
DOI:10.1109/JIOT.2024.3436584