Loading…

Interactions between motor exploration and reinforcement learning

Motor exploration, a trial-and-error process in search for better motor outcomes, is known to serve a critical role in motor learning. This is particularly relevant during reinforcement learning, where actions leading to a successful outcome are reinforced while unsuccessful actions are avoided. Alt...

Full description

Saved in:
Bibliographic Details
Published in:Journal of neurophysiology 2019-08, Vol.122 (2), p.797-808
Main Authors: Uehara, Shintaro, Mawase, Firas, Therrien, Amanda S, Cherry-Allen, Kendra M, Celnik, Pablo
Format: Article
Language:English
Subjects:
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by cdi_FETCH-LOGICAL-c453t-a096f9159c1d23ea99355ea5a566de97e4567f741b10a83ad99085677fbcfb393
cites cdi_FETCH-LOGICAL-c453t-a096f9159c1d23ea99355ea5a566de97e4567f741b10a83ad99085677fbcfb393
container_end_page 808
container_issue 2
container_start_page 797
container_title Journal of neurophysiology
container_volume 122
creator Uehara, Shintaro
Mawase, Firas
Therrien, Amanda S
Cherry-Allen, Kendra M
Celnik, Pablo
description Motor exploration, a trial-and-error process in search for better motor outcomes, is known to serve a critical role in motor learning. This is particularly relevant during reinforcement learning, where actions leading to a successful outcome are reinforced while unsuccessful actions are avoided. Although early on motor exploration is beneficial to finding the correct solution, maintaining high levels of exploration later in the learning process might be deleterious. Whether and how the level of exploration changes over the course of reinforcement learning, however, remains poorly understood. Here we evaluated temporal changes in motor exploration while healthy participants learned a reinforcement-based motor task. We defined exploration as the magnitude of trial-to-trial change in movements as a function of whether the preceding trial resulted in success or failure. Participants were required to find the optimal finger-pointing direction using binary feedback of success or failure. We found that the magnitude of exploration gradually increased over time when participants were learning the task. Conversely, exploration remained low in participants who were unable to correctly adjust their pointing direction. Interestingly, exploration remained elevated when participants underwent a second training session, which was associated with faster relearning. These results indicate that the motor system may flexibly upregulate the extent of exploration during reinforcement learning as if acquiring a specific strategy to facilitate subsequent learning. Also, our findings showed that exploration affects reinforcement learning and vice versa, indicating an interactive relationship between them. Reinforcement-based tasks could be used as primers to increase exploratory behavior leading to more efficient subsequent learning. Motor exploration, the ability to search for the correct actions, is critical to learning motor skills. Despite this, whether and how the level of exploration changes over the course of training remains poorly understood. We showed that exploration increased and remained high throughout training of a reinforcement-based motor task. Interestingly, elevated exploration persisted and facilitated subsequent learning. These results suggest that the motor system upregulates exploration as if learning a strategy to facilitate subsequent learning.
doi_str_mv 10.1152/jn.00390.2018
format article
fullrecord <record><control><sourceid>proquest_pubme</sourceid><recordid>TN_cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_6734400</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2248380524</sourcerecordid><originalsourceid>FETCH-LOGICAL-c453t-a096f9159c1d23ea99355ea5a566de97e4567f741b10a83ad99085677fbcfb393</originalsourceid><addsrcrecordid>eNpVkEtPwzAQhC0EoqVw5Ipy5JKytuM4viBVFY9KSFzgbDnOpqRK7GKnPP49KS0ITrvaGc2OPkLOKUwpFexq5aYAXMGUAS0OyHi4sZQKVRySMcCwc5ByRE5iXAGAFMCOyYhTljHI-ZjMFq7HYGzfeBeTEvt3RJd0vvchwY9164PZSolxVRKwcbUPFjt0fdKiCa5xy1NyVJs24tl-Tsjz7c3T_D59eLxbzGcPqc0E71MDKq_VUMzSinE0SnEh0Agj8rxCJTETuaxlRksKpuCmUgqK4STr0tYlV3xCrne5603ZYWWHDsG0eh2azoRP7U2j_yuuedFL_6ZzybNsYDQhl_uA4F83GHvdNdFi2xqHfhM1Y1nBCxAsG6zpzmqDjzFg_fuGgt5i1yunv7HrLfbBf_G326_7hzP_AvyufxE</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2248380524</pqid></control><display><type>article</type><title>Interactions between motor exploration and reinforcement learning</title><source>American Physiological Society:Jisc Collections:American Physiological Society Journals ‘Read Publish &amp; Join’ Agreement:2023-2024 (Reading list)</source><source>American Physiological Society Free</source><creator>Uehara, Shintaro ; Mawase, Firas ; Therrien, Amanda S ; Cherry-Allen, Kendra M ; Celnik, Pablo</creator><creatorcontrib>Uehara, Shintaro ; Mawase, Firas ; Therrien, Amanda S ; Cherry-Allen, Kendra M ; Celnik, Pablo</creatorcontrib><description>Motor exploration, a trial-and-error process in search for better motor outcomes, is known to serve a critical role in motor learning. This is particularly relevant during reinforcement learning, where actions leading to a successful outcome are reinforced while unsuccessful actions are avoided. Although early on motor exploration is beneficial to finding the correct solution, maintaining high levels of exploration later in the learning process might be deleterious. Whether and how the level of exploration changes over the course of reinforcement learning, however, remains poorly understood. Here we evaluated temporal changes in motor exploration while healthy participants learned a reinforcement-based motor task. We defined exploration as the magnitude of trial-to-trial change in movements as a function of whether the preceding trial resulted in success or failure. Participants were required to find the optimal finger-pointing direction using binary feedback of success or failure. We found that the magnitude of exploration gradually increased over time when participants were learning the task. Conversely, exploration remained low in participants who were unable to correctly adjust their pointing direction. Interestingly, exploration remained elevated when participants underwent a second training session, which was associated with faster relearning. These results indicate that the motor system may flexibly upregulate the extent of exploration during reinforcement learning as if acquiring a specific strategy to facilitate subsequent learning. Also, our findings showed that exploration affects reinforcement learning and vice versa, indicating an interactive relationship between them. Reinforcement-based tasks could be used as primers to increase exploratory behavior leading to more efficient subsequent learning. Motor exploration, the ability to search for the correct actions, is critical to learning motor skills. Despite this, whether and how the level of exploration changes over the course of training remains poorly understood. We showed that exploration increased and remained high throughout training of a reinforcement-based motor task. Interestingly, elevated exploration persisted and facilitated subsequent learning. These results suggest that the motor system upregulates exploration as if learning a strategy to facilitate subsequent learning.</description><identifier>ISSN: 0022-3077</identifier><identifier>EISSN: 1522-1598</identifier><identifier>DOI: 10.1152/jn.00390.2018</identifier><identifier>PMID: 31242063</identifier><language>eng</language><publisher>United States: American Physiological Society</publisher><subject>Adult ; Exploratory Behavior - physiology ; Feedback, Psychological - physiology ; Female ; Humans ; Male ; Metacognition - physiology ; Motor Activity - physiology ; Practice, Psychological ; Psychomotor Performance - physiology ; Reinforcement, Psychology ; Young Adult</subject><ispartof>Journal of neurophysiology, 2019-08, Vol.122 (2), p.797-808</ispartof><rights>Copyright © 2019 the American Physiological Society 2019 American Physiological Society</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c453t-a096f9159c1d23ea99355ea5a566de97e4567f741b10a83ad99085677fbcfb393</citedby><cites>FETCH-LOGICAL-c453t-a096f9159c1d23ea99355ea5a566de97e4567f741b10a83ad99085677fbcfb393</cites><orcidid>0000-0002-6255-1815</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>230,314,780,784,885,27924,27925</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/31242063$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Uehara, Shintaro</creatorcontrib><creatorcontrib>Mawase, Firas</creatorcontrib><creatorcontrib>Therrien, Amanda S</creatorcontrib><creatorcontrib>Cherry-Allen, Kendra M</creatorcontrib><creatorcontrib>Celnik, Pablo</creatorcontrib><title>Interactions between motor exploration and reinforcement learning</title><title>Journal of neurophysiology</title><addtitle>J Neurophysiol</addtitle><description>Motor exploration, a trial-and-error process in search for better motor outcomes, is known to serve a critical role in motor learning. This is particularly relevant during reinforcement learning, where actions leading to a successful outcome are reinforced while unsuccessful actions are avoided. Although early on motor exploration is beneficial to finding the correct solution, maintaining high levels of exploration later in the learning process might be deleterious. Whether and how the level of exploration changes over the course of reinforcement learning, however, remains poorly understood. Here we evaluated temporal changes in motor exploration while healthy participants learned a reinforcement-based motor task. We defined exploration as the magnitude of trial-to-trial change in movements as a function of whether the preceding trial resulted in success or failure. Participants were required to find the optimal finger-pointing direction using binary feedback of success or failure. We found that the magnitude of exploration gradually increased over time when participants were learning the task. Conversely, exploration remained low in participants who were unable to correctly adjust their pointing direction. Interestingly, exploration remained elevated when participants underwent a second training session, which was associated with faster relearning. These results indicate that the motor system may flexibly upregulate the extent of exploration during reinforcement learning as if acquiring a specific strategy to facilitate subsequent learning. Also, our findings showed that exploration affects reinforcement learning and vice versa, indicating an interactive relationship between them. Reinforcement-based tasks could be used as primers to increase exploratory behavior leading to more efficient subsequent learning. Motor exploration, the ability to search for the correct actions, is critical to learning motor skills. Despite this, whether and how the level of exploration changes over the course of training remains poorly understood. We showed that exploration increased and remained high throughout training of a reinforcement-based motor task. Interestingly, elevated exploration persisted and facilitated subsequent learning. These results suggest that the motor system upregulates exploration as if learning a strategy to facilitate subsequent learning.</description><subject>Adult</subject><subject>Exploratory Behavior - physiology</subject><subject>Feedback, Psychological - physiology</subject><subject>Female</subject><subject>Humans</subject><subject>Male</subject><subject>Metacognition - physiology</subject><subject>Motor Activity - physiology</subject><subject>Practice, Psychological</subject><subject>Psychomotor Performance - physiology</subject><subject>Reinforcement, Psychology</subject><subject>Young Adult</subject><issn>0022-3077</issn><issn>1522-1598</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2019</creationdate><recordtype>article</recordtype><recordid>eNpVkEtPwzAQhC0EoqVw5Ipy5JKytuM4viBVFY9KSFzgbDnOpqRK7GKnPP49KS0ITrvaGc2OPkLOKUwpFexq5aYAXMGUAS0OyHi4sZQKVRySMcCwc5ByRE5iXAGAFMCOyYhTljHI-ZjMFq7HYGzfeBeTEvt3RJd0vvchwY9164PZSolxVRKwcbUPFjt0fdKiCa5xy1NyVJs24tl-Tsjz7c3T_D59eLxbzGcPqc0E71MDKq_VUMzSinE0SnEh0Agj8rxCJTETuaxlRksKpuCmUgqK4STr0tYlV3xCrne5603ZYWWHDsG0eh2azoRP7U2j_yuuedFL_6ZzybNsYDQhl_uA4F83GHvdNdFi2xqHfhM1Y1nBCxAsG6zpzmqDjzFg_fuGgt5i1yunv7HrLfbBf_G326_7hzP_AvyufxE</recordid><startdate>20190801</startdate><enddate>20190801</enddate><creator>Uehara, Shintaro</creator><creator>Mawase, Firas</creator><creator>Therrien, Amanda S</creator><creator>Cherry-Allen, Kendra M</creator><creator>Celnik, Pablo</creator><general>American Physiological Society</general><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7X8</scope><scope>5PM</scope><orcidid>https://orcid.org/0000-0002-6255-1815</orcidid></search><sort><creationdate>20190801</creationdate><title>Interactions between motor exploration and reinforcement learning</title><author>Uehara, Shintaro ; Mawase, Firas ; Therrien, Amanda S ; Cherry-Allen, Kendra M ; Celnik, Pablo</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c453t-a096f9159c1d23ea99355ea5a566de97e4567f741b10a83ad99085677fbcfb393</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2019</creationdate><topic>Adult</topic><topic>Exploratory Behavior - physiology</topic><topic>Feedback, Psychological - physiology</topic><topic>Female</topic><topic>Humans</topic><topic>Male</topic><topic>Metacognition - physiology</topic><topic>Motor Activity - physiology</topic><topic>Practice, Psychological</topic><topic>Psychomotor Performance - physiology</topic><topic>Reinforcement, Psychology</topic><topic>Young Adult</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Uehara, Shintaro</creatorcontrib><creatorcontrib>Mawase, Firas</creatorcontrib><creatorcontrib>Therrien, Amanda S</creatorcontrib><creatorcontrib>Cherry-Allen, Kendra M</creatorcontrib><creatorcontrib>Celnik, Pablo</creatorcontrib><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>MEDLINE - Academic</collection><collection>PubMed Central (Full Participant titles)</collection><jtitle>Journal of neurophysiology</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Uehara, Shintaro</au><au>Mawase, Firas</au><au>Therrien, Amanda S</au><au>Cherry-Allen, Kendra M</au><au>Celnik, Pablo</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Interactions between motor exploration and reinforcement learning</atitle><jtitle>Journal of neurophysiology</jtitle><addtitle>J Neurophysiol</addtitle><date>2019-08-01</date><risdate>2019</risdate><volume>122</volume><issue>2</issue><spage>797</spage><epage>808</epage><pages>797-808</pages><issn>0022-3077</issn><eissn>1522-1598</eissn><abstract>Motor exploration, a trial-and-error process in search for better motor outcomes, is known to serve a critical role in motor learning. This is particularly relevant during reinforcement learning, where actions leading to a successful outcome are reinforced while unsuccessful actions are avoided. Although early on motor exploration is beneficial to finding the correct solution, maintaining high levels of exploration later in the learning process might be deleterious. Whether and how the level of exploration changes over the course of reinforcement learning, however, remains poorly understood. Here we evaluated temporal changes in motor exploration while healthy participants learned a reinforcement-based motor task. We defined exploration as the magnitude of trial-to-trial change in movements as a function of whether the preceding trial resulted in success or failure. Participants were required to find the optimal finger-pointing direction using binary feedback of success or failure. We found that the magnitude of exploration gradually increased over time when participants were learning the task. Conversely, exploration remained low in participants who were unable to correctly adjust their pointing direction. Interestingly, exploration remained elevated when participants underwent a second training session, which was associated with faster relearning. These results indicate that the motor system may flexibly upregulate the extent of exploration during reinforcement learning as if acquiring a specific strategy to facilitate subsequent learning. Also, our findings showed that exploration affects reinforcement learning and vice versa, indicating an interactive relationship between them. Reinforcement-based tasks could be used as primers to increase exploratory behavior leading to more efficient subsequent learning. Motor exploration, the ability to search for the correct actions, is critical to learning motor skills. Despite this, whether and how the level of exploration changes over the course of training remains poorly understood. We showed that exploration increased and remained high throughout training of a reinforcement-based motor task. Interestingly, elevated exploration persisted and facilitated subsequent learning. These results suggest that the motor system upregulates exploration as if learning a strategy to facilitate subsequent learning.</abstract><cop>United States</cop><pub>American Physiological Society</pub><pmid>31242063</pmid><doi>10.1152/jn.00390.2018</doi><tpages>12</tpages><orcidid>https://orcid.org/0000-0002-6255-1815</orcidid><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 0022-3077
ispartof Journal of neurophysiology, 2019-08, Vol.122 (2), p.797-808
issn 0022-3077
1522-1598
language eng
recordid cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_6734400
source American Physiological Society:Jisc Collections:American Physiological Society Journals ‘Read Publish & Join’ Agreement:2023-2024 (Reading list); American Physiological Society Free
subjects Adult
Exploratory Behavior - physiology
Feedback, Psychological - physiology
Female
Humans
Male
Metacognition - physiology
Motor Activity - physiology
Practice, Psychological
Psychomotor Performance - physiology
Reinforcement, Psychology
Young Adult
title Interactions between motor exploration and reinforcement learning
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-06T07%3A22%3A08IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_pubme&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Interactions%20between%20motor%20exploration%20and%20reinforcement%20learning&rft.jtitle=Journal%20of%20neurophysiology&rft.au=Uehara,%20Shintaro&rft.date=2019-08-01&rft.volume=122&rft.issue=2&rft.spage=797&rft.epage=808&rft.pages=797-808&rft.issn=0022-3077&rft.eissn=1522-1598&rft_id=info:doi/10.1152/jn.00390.2018&rft_dat=%3Cproquest_pubme%3E2248380524%3C/proquest_pubme%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c453t-a096f9159c1d23ea99355ea5a566de97e4567f741b10a83ad99085677fbcfb393%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2248380524&rft_id=info:pmid/31242063&rfr_iscdi=true