Embodied Synaptic Plasticity With Online Reinforcement Learning

The endeavor to understand the brain involves multiple collaborating research fields. Classically, synaptic plasticity rules derived by theoretical neuroscientists are evaluated in isolation on pattern classification tasks. This contrasts with the biological brain which purpose is to control a body...

Full description

Saved in:
Bibliographic Details
Published in:Frontiers in neurorobotics 2019-10, Vol.13, p.81-81
Main Authors: Kaiser, Jacques, Hoff, Michael, Konle, Andreas, Vasquez Tieck, J. Camilo, Kappel, David, Reichard, Daniel, Subramoney, Anand, Legenstein, Robert, Roennau, Arne, Maass, Wolfgang, Dillmann, Rüdiger
Format: Article
Language:English
Subjects:
Algorithms
Back propagation
Computational neuroscience
Deep learning
Dopamine
Environmental impact
International conferences
Internet
Learning
Nervous system
Network management systems
Neural networks
neuromorphic vision
Neurons
neurorobotics
Neuroscience
Neurosciences
Open source software
Optimization
Public domain
Reinforcement
reinforcement learning
Robotics
Sensorimotor integration
spiking neural networks
Synaptic plasticity
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The endeavor to understand the brain involves multiple collaborating research fields. Classically, synaptic plasticity rules derived by theoretical neuroscientists are evaluated in isolation on pattern classification tasks. This contrasts with the biological brain which purpose is to control a body in closed-loop. This paper contributes to bringing the fields of computational neuroscience and robotics closer together by integrating open-source software components from these two fields. The resulting framework allows to evaluate the validity of biologically-plausibe plasticity models in closed-loop robotics environments. We demonstrate this framework to evaluate Synaptic Plasticity with Online REinforcement learning (SPORE), a reward-learning rule based on synaptic sampling, on two visuomotor tasks: reaching and lane following. We show that SPORE is capable of learning to perform policies within the course of simulated hours for both tasks. Provisional parameter explorations indicate that the learning rate and the temperature driving the stochastic processes that govern synaptic learning dynamics need to be regulated for performance improvements to be retained. We conclude by discussing the recent deep reinforcement learning techniques which would be beneficial to increase the functionality of SPORE on visuomotor tasks.
ISSN:1662-5218
1662-5218
DOI:10.3389/fnbot.2019.00081