Uncertainty in learning, choice, and visual fixation

Uncertainty plays a critical role in reinforcement learning and decision making. However, exactly how it influences behavior remains unclear. Multiarmed-bandit tasks offer an ideal test bed, since computational tools such as approximate Kalman filters can closely characterize the interplay between t...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings of the National Academy of Sciences - PNAS 2020-02, Vol.117 (6), p.3291-3300
Main Authors: Stojić, Hrvoje, Orquin, Jacob L., Dayan, Peter, Dolan, Raymond J., Speekenbrink, Maarten
Format: Article
Language:English
Subjects:
Adolescent
Adult
Algorithms
Biological Sciences
Choice learning
Computer applications
Decision making
Decision Making - physiology
Exploratory Behavior - physiology
Female
Fixation
Fixation, Ocular - physiology
Humans
Kalman filters
Learning
Male
Reinforcement, Psychology
Social Sciences
Software
Task Performance and Analysis
Uncertainty
Visual discrimination learning
Young Adult
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Uncertainty plays a critical role in reinforcement learning and decision making. However, exactly how it influences behavior remains unclear. Multiarmed-bandit tasks offer an ideal test bed, since computational tools such as approximate Kalman filters can closely characterize the interplay between trial-by-trial values, uncertainty, learning, and choice. To gain additional insight into learning and choice processes, we obtained data from subjects’ overt allocation of gaze. The estimated value and estimation uncertainty of options influenced what subjects looked at before choosing; these same quantities also influenced choice, as additionally did fixation itself. A momentary measure of uncertainty in the form of absolute prediction errors determined how long participants looked at the obtained outcomes. These findings affirm the importance of uncertainty in multiple facets of behavior and help delineate its effects on decision making.
ISSN:0027-8424
1091-6490
DOI:10.1073/pnas.1911348117