Multisensory Proximity and Transition Cues for Improving Target Awareness in Narrow Field of View Augmented Reality Displays

• Published in: IEEE transactions on visualization and computer graphics 2022-02, Vol.28 (2), p.1342-1362
• Main Authors: Trepkowski, Christina, Marquardt, Alexander, Eibich, Tom David, Shikanai, Yusuke, Maiero, Jens, Kiyokawa, Kiyoshi, Kruijff, Ernst, Schoning, Johannes, Konig, Peter
• Format: Article
• Language: English
• Subjects: Augmented Reality; Clutter; Computer Graphics; Cues; Displays; Field of view; guidance; multisensory cues; Noise levels; Occlusion; Proximity; Search problems; situation awareness; Task analysis; Three-dimensional displays; Touch; Urban areas; view management; Visual effects; Visual fields; Visual Perception; Visualization; Working environment noise
• ISSN: 1077-2626; 1941-0506
• DOI: 10.1109/TVCG.2021.3116673

Augmented reality applications allow users to enrich their real surroundings with additional digital content. However, due to the limited field of view of augmented reality devices, it can sometimes be difficult to become aware of newly emerging information inside or outside the field of view. Typical visual conflicts like clutter and occlusion of augmentations occur and can be further aggravated especially in the context of dense information spaces. In this article, we evaluate how multisensory cue combinations can improve the awareness for moving out-of-view objects in narrow field of view augmented reality displays. We distinguish between proximity and transition cues in either visual, auditory or tactile manner. Proximity cues are intended to enhance spatial awareness of approaching out-of-view objects while transition cues inform the user that the object just entered the field of view. In study 1, user preference was determined for 6 different cue combinations via forced-choice decisions. In study 2, the 3 most preferred modes were then evaluated with respect to performance and awareness measures in a divided attention reaction task. Both studies were conducted under varying noise levels. We show that on average the Visual-Tactile combination leads to 63% and Audio-Tactile to 65% faster reactions to incoming out-of-view augmentations than their Visual-Audio counterpart, indicating a high usefulness of tactile transition cues. We further show a detrimental effect of visual and audio noise on performance when feedback included visual proximity cues. Based on these results, we make recommendations to determine which cue combination is appropriate for which application.