Viiat-Hand: A Reach-and-Grasp Restoration System Integrating Voice Interaction, Computer Vision, Auditory and Tactile Feedback for Non-Sighted Amputees

For non-sighted and visually impaired (BVI) amputees, the combined loss of vision and grasping abilities turns the seemingly simple task of reaching and grasping into a significant challenge. This letter introduces a novel multi-sensory prosthesis system designed for BVI amputees to assist in percep...

Full description

Saved in:
Bibliographic Details
Published in:IEEE robotics and automation letters 2024-10, Vol.9 (10), p.8674-8681
Main Authors: Peng, Chunhao, Yang, Dapeng, Zhao, Deyu, Cheng, Ming, Dai, Jinghui, Jiang, Li
Format: Article
Language:English
Subjects:
auditory aid
Cameras
Collaboration
computer vision
human-machine interaction
Navigation
Prosthetic hand
Prosthetics
Task analysis
Vibrations
visual impairment
Wrist
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:For non-sighted and visually impaired (BVI) amputees, the combined loss of vision and grasping abilities turns the seemingly simple task of reaching and grasping into a significant challenge. This letter introduces a novel multi-sensory prosthesis system designed for BVI amputees to assist in perception, navigation, and grasping tasks. The system integrates voice interaction, environmental perception, grasp guidance, collaborative control, and auditory/tactile feedback. Specifically, it processes user commands, provides environmental data via auditory/tactile channels, and manages collaborative control of grasp gestures and wrist angles for stable object handling. The prototype, viiat-hand, was experimentally tested with eight non-disabled and four non-sighted subjects performing reach-and-grasp tasks, showing that users could accurately reach (average time:15.24 s) and securely grasp objects (average time:17.23 s) in an indoor setting. The system also proved to be user-friendly, requiring minimal training for users to become adept.
ISSN:2377-3766
2377-3766
DOI:10.1109/LRA.2024.3448218