Eyes and ears: Using eye tracking and pupillometry to understand challenges to speech recognition

Although human speech recognition is often experienced as relatively effortless, a number of common challenges can render the task more difficult. Such challenges may originate in talkers (e.g., unfamiliar accents, varying speech styles), the environment (e.g. noise), or in listeners themselves (e.g...

Full description

Saved in:
Bibliographic Details
Published in:Hearing research 2018-11, Vol.369, p.56-66
Main Authors: Van Engen, Kristin J., McLaughlin, Drew J.
Format: Article
Language:English
Subjects:
Age Factors
Aging - psychology
Attention
Cues
Eye Movements
Eye tracking
Hearing
Hearing Disorders - physiopathology
Hearing Disorders - psychology
Humans
Listening effort
Pupil - physiology
Pupillometry
Recognition, Psychology
Speech Intelligibility
Speech Perception
Speech recognition
Time Factors
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Although human speech recognition is often experienced as relatively effortless, a number of common challenges can render the task more difficult. Such challenges may originate in talkers (e.g., unfamiliar accents, varying speech styles), the environment (e.g. noise), or in listeners themselves (e.g., hearing loss, aging, different native language backgrounds). Each of these challenges can reduce the intelligibility of spoken language, but even when intelligibility remains high, they can place greater processing demands on listeners. Noisy conditions, for example, can lead to poorer recall for speech, even when it has been correctly understood. Speech intelligibility measures, memory tasks, and subjective reports of listener difficulty all provide critical information about the effects of such challenges on speech recognition. Eye tracking and pupillometry complement these methods by providing objective physiological measures of online cognitive processing during listening. Eye tracking records the moment-to-moment direction of listeners' visual attention, which is closely time-locked to unfolding speech signals, and pupillometry measures the moment-to-moment size of listeners' pupils, which dilate in response to increased cognitive load. In this paper, we review the uses of these two methods for studying challenges to speech recognition. •Reviews the uses of eye tracking and pupillometry for studying challenges to speech recognition.
ISSN:0378-5955
1878-5891
DOI:10.1016/j.heares.2018.04.013