Asymmetries in Unimodal Visual Vowel Perception: The Roles of Oral-Facial Kinematics, Orientation, and Configuration

• Published in: Journal of experimental psychology. Human perception and performance 2018-07, Vol.44 (7), p.1103-1118
• Main Authors: Masapollo, Matthew, Polka, Linda, Ménard, Lucie, Franklin, Lauren, Tiede, Mark, Morgan, James
• Format: Article
• Language: English
• Subjects: Adult; Asymmetry; Auditory discrimination; Bias; Convergence; Discrimination; Eye Movement Measurements; Female; Gestures; Human; Humans; Kinematics; Male; Mouth; Mouth - physiology; Orientation behavior; Psycholinguistics; Sensory perception; Spatial discrimination; Speech; Speech - physiology; Speech Perception; Speech Perception - physiology; Visual communication; Visual discrimination; Visual Perception; Visual Perception - physiology; Vowels; Young Adult
• ISSN: 0096-1523; 1939-1277
• DOI: 10.1037/xhp0000518

Masapollo, Polka, and Ménard (2017) recently reported a robust directional asymmetry in unimodal visual vowel perception: Adult perceivers discriminate a change from an English /u/ viseme to a French /u/ viseme significantly better than a change in the reverse direction. This asymmetry replicates a frequent pattern found in unimodal auditory vowel perception that points to a universal bias favoring more extreme vocalic articulations, which lead to acoustic signals with increased formant convergence. In the present article, the authors report 5 experiments designed to investigate whether this asymmetry in the visual realm reflects a speech-specific or general processing bias. They successfully replicated the directional effect using Masapollo et al.'s dynamically articulating faces but failed to replicate the effect when the faces were shown under static conditions. Asymmetries also emerged during discrimination of canonically oriented point-light stimuli that retained the kinematics and configuration of the articulating mouth. In contrast, no asymmetries emerged during discrimination of rotated point-light stimuli or Lissajou patterns that retained the kinematics, but not the canonical orientation or spatial configuration, of the labial gestures. These findings suggest that the perceptual processes underlying asymmetries in unimodal visual vowel discrimination are sensitive to speech-specific motion and configural properties and raise foundational questions concerning the role of specialized and general processes in vowel perception. Public Significance Statement The present research explores the precise nature of the visual speech information that adults perceive during face-to-face conversational interactions by examining how adults discriminate unimodal visual vowels and schematic visual analogs of them that track lip movements during vowel production but are not perceived as speech. We examined a well-established perceptual bias favoring vowels produced with extreme vocal tract maneuvers, which can be measured as a directional asymmetry in auditory or visual vowel discrimination performance. Results show that adults display this bias when discriminating visual-only vowels and attend closely to the mouth during the discrimination task. Adults also display the bias when discriminating visual stimuli that capture both the shape and movement of a talking mouth producing vowels but not when either the shape or the movement is disturbed. Collectively, the fi