Emergence of Prediction by Reinforcement Learning Using a Recurrent Neural Network

• Published in: Journal of robotics 2010-01, Vol.2010 (2010), p.1-9
• Main Authors: Goto, Kenta, Shibata, Katsunari
• Format: Article
• Language: English
• ISSN: 1687-9600; 1687-9619
• DOI: 10.1155/2010/437654

To develop a robot that behaves flexibly in the real world, it is essential that it learns various necessary functions autonomously without receiving significant information from a human in advance. Among such functions, this paper focuses on learning “prediction” that is attracting attention recently from the viewpoint of autonomous learning. The authors point out that it is important to acquire through learning not only the way of predicting future information, but also the purposive extraction of prediction target from sensor signals. It is suggested that through reinforcement learning using a recurrent neural network, both emerge purposively and simultaneously without testing individually whether or not each piece of information is predictable. In a task where an agent gets a reward when it catches a moving object that can possibly become invisible, it was observed that the agent learned to detect the necessary factors of the object velocity before it disappeared, to relay the information among some hidden neurons, and finally to catch the object at an appropriate position and timing, considering the effects of bounces off a wall after the object became invisible.