A deep hierarchy of predictions enables online meaning extraction in a computational model of human speech comprehension

Understanding speech requires mapping fleeting and often ambiguous soundwaves to meaning. While humans are known to exploit their capacity to contextualize to facilitate this process, how internal knowledge is deployed online remains an open question. Here, we present a model that extracts multiple...

Full description

Saved in:
Bibliographic Details
Published in:PLoS biology 2023-03, Vol.21 (3), p.e3002046
Main Authors: Su, Yaqing, MacGregor, Lucy J, Olasagasti, Itsaso, Giraud, Anne-Lise
Format: Article
Language:English
Subjects:
Acoustics
Analysis
Biology and Life Sciences
Brain - physiology
Cognitive science
Cognitive Sciences
Comprehension - physiology
Computational neuroscience
Computer simulation
Computer-generated environments
Engineering and Technology
Flexibility
Humans
Knowledge
Language
Life Sciences
Linguistics
Neural networks
Neurons and Cognition
Neuroscience
Physical Sciences
Predictions
Semantics
Social Sciences
Speech
Speech Perception - physiology
Speech processing
Syntax
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Understanding speech requires mapping fleeting and often ambiguous soundwaves to meaning. While humans are known to exploit their capacity to contextualize to facilitate this process, how internal knowledge is deployed online remains an open question. Here, we present a model that extracts multiple levels of information from continuous speech online. The model applies linguistic and nonlinguistic knowledge to speech processing, by periodically generating top-down predictions and incorporating bottom-up incoming evidence in a nested temporal hierarchy. We show that a nonlinguistic context level provides semantic predictions informed by sensory inputs, which are crucial for disambiguating among multiple meanings of the same word. The explicit knowledge hierarchy of the model enables a more holistic account of the neurophysiological responses to speech compared to using lexical predictions generated by a neural network language model (GPT-2). We also show that hierarchical predictions reduce peripheral processing via minimizing uncertainty and prediction error. With this proof-of-concept model, we demonstrate that the deployment of hierarchical predictions is a possible strategy for the brain to dynamically utilize structured knowledge and make sense of the speech input.
ISSN:1545-7885
1544-9173
1545-7885
DOI:10.1371/journal.pbio.3002046