RASPV: A Robotics Framework for Augmented Simulated Prosthetic Vision

One of the main challenges of visual prostheses is to augment the perceived information to improve the experience of its wearers. Given the limited access to implanted patients, in order to facilitate the experimentation of new techniques, this is often evaluated via Simulated Prosthetic Vision (SPV...

Full description

Saved in:
Bibliographic Details
Published in:IEEE access 2024, Vol.12, p.1-1
Main Authors: Perez-Yus, Alejandro, Santos-Villafranca, Maria, Tomas-Barba, Julia, Bermudez-Cameo, Jesus, Montano-Olivan, Lorenzo, Lopez-Nicolas, Gonzalo, Guerrero, Jose J.
Format: Article
Language:English
Subjects:
Algorithms
Artificial vision
Autonomous navigation
Cameras
Computer vision
Experimentation
Helmet mounted displays
Image processing
Machine learning
Moving obstacles
navigation
Neural prostheses
Object recognition
Obstacle avoidance
Phosphene
Prostheses
RGB-D
Robotics
simulated prosthetic vision
Three dimensional models
Virtual environments
Visual stimuli
visually impaired assistance
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:One of the main challenges of visual prostheses is to augment the perceived information to improve the experience of its wearers. Given the limited access to implanted patients, in order to facilitate the experimentation of new techniques, this is often evaluated via Simulated Prosthetic Vision (SPV) with sighted people. In this work, we introduce a novel SPV framework and implementation that presents major advantages with respect to previous approaches. First, it is integrated into a robotics framework, which allows us to benefit from a wide range of methods and algorithms from the field (e.g. object recognition, obstacle avoidance, autonomous navigation, deep learning). Second, we go beyond traditional image processing with 3D point clouds processing using an RGB-D camera, allowing us to robustly detect the floor, obstacles and the structure of the scene. Third, it works either with a real camera or in a virtual environment, which gives us endless possibilities for immersive experimentation through a head-mounted display. Fourth, we incorporate a validated temporal phosphene model that replicates time effects into the generation of visual stimuli. Finally, we have proposed, developed and tested several applications within this framework, such as avoiding moving obstacles, providing a general understanding of the scene, staircase detection, helping the subject to navigate an unfamiliar space, and object and person detection. We provide experimental results in real and virtual environments. The code is publicly available at https://www.github.com/aperezyus/RASPV
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2024.3357400