Wearable Travel Aid for Environment Perception and Navigation of Visually Impaired People

Assistive devices for visually impaired people (VIP) which support daily traveling and improve social inclusion are developing fast. Most of them try to solve the problem of navigation or obstacle avoidance, and other works focus on helping VIP to recognize their surrounding objects. However, very f...

Full description

Saved in:
Bibliographic Details
Published in:Electronics (Basel) 2019-06, Vol.8 (6), p.697
Main Authors: Bai, Jinqiang, Liu, Zhaoxiang, Lin, Yimin, Li, Ye, Lian, Shiguo, Liu, Dijun
Format: Article
Language:English
Subjects:
Artificial neural networks
Blindness
Design
Eyewear
Human performance
Indoor environments
Inertial platforms
Navigation systems
Object recognition
Obstacle avoidance
People with disabilities
Perception
Semantics
Sensors
Smartphones
Speech recognition
Visual impairment
Wearable technology
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Assistive devices for visually impaired people (VIP) which support daily traveling and improve social inclusion are developing fast. Most of them try to solve the problem of navigation or obstacle avoidance, and other works focus on helping VIP to recognize their surrounding objects. However, very few of them couple both capabilities (i.e., navigation and recognition). Aiming at the above needs, this paper presents a wearable assistive device that allows VIP to (i) navigate safely and quickly in unfamiliar environment, and (ii) to recognize the objects in both indoor and outdoor environments. The device consists of a consumer Red, Green, Blue and Depth (RGB-D) camera and an Inertial Measurement Unit (IMU), which are mounted on a pair of eyeglasses, and a smartphone. The device leverages the ground height continuity among adjacent image frames to segment the ground accurately and rapidly, and then search the moving direction according to the ground. A lightweight Convolutional Neural Network (CNN)-based object recognition system is developed and deployed on the smartphone to increase the perception ability of VIP and promote the navigation system. It can provide the semantic information of surroundings, such as the categories, locations, and orientations of objects. Human–machine interaction is performed through audio module (a beeping sound for obstacle alert, speech recognition for understanding the user commands, and speech synthesis for expressing semantic information of surroundings). We evaluated the performance of the proposed system through many experiments conducted in both indoor and outdoor scenarios, demonstrating the efficiency and safety of the proposed assistive system.
ISSN:2079-9292
2079-9292
DOI:10.3390/electronics8060697