Word meaning and the control of eye fixation: semantic competitor effects and the visual world paradigm

• Published in: Cognition 2005-05, Vol.96 (1), p.B23-B32
• Main Authors: Huettig, Falk, Altmann, Gerry T.M.
• Format: Article
• Language: English
• Subjects: Association Learning; Attention; Auditory Perception; Biological and medical sciences; Cognitive Psychology; Comprehension; Concept Formation; Eye Movements; Fixation, Ocular; Fundamental and applied biological sciences. Psychology; Human Body; Humans; Language; Language Processing; Meaning; Oral Language; Orientation; Pattern Recognition, Visual; Priming; Probability; Production and perception of spoken language; Psychological effects; Psychology. Psychoanalysis. Psychiatry; Psychology. Psychophysiology; Psychophysics; Saccades; Semantic features; Semantics; Speech; Speech Perception; Visual perception; Visual world; Words
• ISSN: 0010-0277; 1873-7838
• DOI: 10.1016/j.cognition.2004.10.003

When participants are presented simultaneously with spoken language and a visual display depicting objects to which that language refers, participants spontaneously fixate the visual referents of the words being heard [Cooper, R. M. (1974). The control of eye fixation by the meaning of spoken language: A new methodology for the real-time investigation of speech perception, memory, and language processing. Cognitive Psychology, 6(1), 84–107; Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268(5217), 1632–1634]. We demonstrate here that such spontaneous fixation can be driven by partial semantic overlap between a word and a visual object. Participants heard the word ‘piano’ when (a) a piano was depicted amongst unrelated distractors; (b) a trumpet was depicted amongst those same distractors; and (c), both the piano and trumpet were depicted. The probability of fixating the piano and the trumpet in the first two conditions rose as the word ‘piano’ unfolded. In the final condition, only fixations to the piano rose, although the trumpet was fixated more than the distractors. We conclude that eye movements are driven by the degree of match, along various dimensions that go beyond simple visual form, between a word and the mental representations of objects in the concurrent visual field.