Differential reinforcement encoding along the hippocampal long axis helps resolve the explore–exploit dilemma

• Published in: Nature communications 2020-10, Vol.11 (1), p.5407-5407, Article 5407
• Main Authors: Dombrovski, Alexandre Y., Luna, Beatriz, Hallquist, Michael N.
• Format: Article
• Language: English
• Subjects: 59/36; 631/378/1595/1554; 631/378/1788; 631/378/2649/1409; Adolescent; Adult; Amygdala; Cerebral cortex; Choice Behavior; Cognitive ability; Cognitive maps; Cognitive models; Cognitive tasks; Decision Making; Decisions; Error signals; Expected values; Exploitation; Female; Functional magnetic resonance imaging; Hippocampus; Hippocampus - physiology; Humanities and Social Sciences; Humans; Learning; Male; multidisciplinary; Neocortex; Neostriatum; Reinforcement; Reinforcement, Psychology; Reward; Science; Science (multidisciplinary); Young Adult
• ISSN: 2041-1723; 2041-1723
• DOI: 10.1038/s41467-020-18864-0

When making decisions, should one exploit known good options or explore potentially better alternatives? Exploration of spatially unstructured options depends on the neocortex, striatum, and amygdala. In natural environments, however, better options often cluster together, forming structured value distributions. The hippocampus binds reward information into allocentric cognitive maps to support navigation and foraging in such spaces. Here we report that human posterior hippocampus (PH) invigorates exploration while anterior hippocampus (AH) supports the transition to exploitation on a reinforcement learning task with a spatially structured reward function. These dynamics depend on differential reinforcement representations in the PH and AH. Whereas local reward prediction error signals are early and phasic in the PH tail, global value maximum signals are delayed and sustained in the AH body. AH compresses reinforcement information across episodes, updating the location and prominence of the value maximum and displaying goal cell-like ramping activity when navigating toward it. Decisions under uncertainty involve a balance between exploiting familiar valuable options and exploring unfamiliar ones. Here, the authors study hippocampal responses using fMRI during a reinforcement learning task, and show the differential involvement of the anterior-posterior regions in the explore-exploit aspects of the task.