Input modality matters: A comparison of touch, speech, and gesture based in-vehicle interaction

• Published in: Applied ergonomics 2023-04, Vol.108, p.103958, Article 103958
• Main Authors: Zhang, Tingru, Liu, Xing, Zeng, Weisheng, Tao, Da, Li, Guofa, Qu, Xingda
• Format: Article
• Language: English
• Subjects: Accidents, Traffic; Automobile Driving; Driver distraction; Gesture; Gestures; Humans; In-vehicle information system; Reaction Time; Speech; Touchscreen
• ISSN: 0003-6870; 1872-9126
• DOI: 10.1016/j.apergo.2022.103958

Innovative input devices are being available for in-vehicle information systems (IVISs). While they have the potential to provide enjoyable driving by enabling drivers to perform non-driving related tasks (NDRTs) in more natural ways, the associated distracting effects should be paid with more attention. The purpose of this exploratory study was to compare the effects of three novel input modalities, i.e., touchscreen-based interaction (TBI), speech-based interaction (SBI), and gesture-based interaction (GBI), on driving performance and driver visual behaviors. Moreover, we examined if the influence of different modalities would be moderated by the difficulty level of NDRTs. A total of 36 participants were invited to a simulated driving experiment where they were randomly assigned to one of the four groups (TBI, GBI, SBI or baseline) and completed three driving trials. The results showed that TBI led to the worse driving performance, as indicated by the significantly prolonged reaction time, reduced minimum time-to-collision, and increased variations in both longitudinal and lateral vehicle control. The deteriorated driving performance could be attributed, at least partially, to the intense visual demand induced by looking towards the touchscreen, as indicated by more and longer off-the-road glances. The adverse impacts of GBI were relatively smaller, but it still posed great crash risk by leading to a shorter minimum time-to-collision and less stable vehicle control compared to the baseline. SBI, although not completely equivalent to the baseline group, showed the minimum influence on driving and visual performance. Only very few interaction effects were found, suggesting that the effects of modality were quite robust across different NDRTs. It was concluded that SBI and GBI provided safer alternatives to in-vehicle interaction than TBI. •The effects of three novel input modalities on driving and visual performance are investigated.•Touchscreen-based interaction leads to the worse driving performance and the largest number of off-the-road glances.•Gesture-based interaction deteriorates reaction time and driving stability, but does not impair safety-critical glance.•Speech-based interaction shows the minimum adverse influence, but still is not completely equivalent to the baseline.•The interaction effects between modality and task difficulty are limited.