Semantic and structural image segmentation for prosthetic vision

• Published in: PloS one 2020-01, Vol.15 (1), p.e0227677-e0227677
• Main Authors: Sanchez-Garcia, Melani, Martinez-Cantin, Ruben, Guerrero, Jose J
• Format: Article
• Language: English
• Subjects: Adult; Artificial Intelligence - statistics & numerical data; Artificial neural networks; Biology and Life Sciences; Cable television broadcasting industry; Cameras; Color vision; Computer Simulation; Computer vision; Deep learning; Disabled people; Engineering and Technology; Experiments; Female; Healthy Volunteers; Humans; Image processing; Image Processing, Computer-Assisted; Image segmentation; Indoor environments; Male; Medicine and Health Sciences; Methods; Middle Aged; Neural networks; Object recognition; Pattern recognition; People with disabilities; Perceptions; Phosphene; Phosphenes - physiology; Photic Stimulation - methods; Prostheses; Prostheses and implants; Psychophysics; Research and Analysis Methods; Retina; Scene analysis; Segmentation; Semantics; Sensory stimulation; Social Sciences; Software; Surgical implants; Transplants & implants; Vision; Vision Disorders - physiopathology; Vision Disorders - psychology; Vision Disorders - therapy; Vision systems; Visual Perception; Visual Prosthesis - statistics & numerical data; Young Adult
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0227677

Prosthetic vision is being applied to partially recover the retinal stimulation of visually impaired people. However, the phosphenic images produced by the implants have very limited information bandwidth due to the poor resolution and lack of color or contrast. The ability of object recognition and scene understanding in real environments is severely restricted for prosthetic users. Computer vision can play a key role to overcome the limitations and to optimize the visual information in the prosthetic vision, improving the amount of information that is presented. We present a new approach to build a schematic representation of indoor environments for simulated phosphene images. The proposed method combines a variety of convolutional neural networks for extracting and conveying relevant information about the scene such as structural informative edges of the environment and silhouettes of segmented objects. Experiments were conducted with normal sighted subjects with a Simulated Prosthetic Vision system. The results show good accuracy for object recognition and room identification tasks for indoor scenes using the proposed approach, compared to other image processing methods.