Interactive Image Exploration for Visually Impaired Readers using Audio-augmented Touch Gestures

• Main Authors: Kruger, Rynhardt, Wet, Febe de, Niesler, Thomas
• Format: Conference Proceeding
• Language: English
• Subjects: accessibility; blindness; diagrams; Rendering (computer graphics); Sensory substitution; Shape; Sociology; Statistics; Touch sensitive screens; visual impairment; Visualization; vOICe algorithm
• ISSN: 2375-0138
• DOI: 10.1109/IV51561.2020.00093

Technologies such as text to speech and hardware braille displays provide alternative representations of electronic text. As a result, electronic documents provide increased accessibility to print-disabled users. However, non-textual graphical information in electronic documents remain largely inaccessible to the blind population. In this study, we explore audio-visual sensory substitution as a means of rendering graphical information present in electronic documents to blind users. To achieve this, we have extended the audio rendering approach used by the well-established vOICe algorithm to allow interactive and localised exploration of an image by means of gestures and the touch screen of a standard commercially-available tablet. The effectiveness of our approach was evaluated in a set of user trials that required six sighted and six blind subjects to identify elements of scenes consisting of a number of geometrical shapes and emoticons. Our results show that both groups of subjects were more successful at identifying shapes using the interactive algorithm than with the baseline vOICe algorithm to a highly statistically significant degree. Furthermore, the results indicate that this improvement is greatest for the most complex scenes. We conclude that, by introducing an interactive touch interface, the vOICe algorithm can be successfully extended to allow interactive exploration and interpretation of diagrams, thereby improving accessibility to material such as scientific publications.