Deep Reinforcement Learning for Humanoid Robot Behaviors

• Published in: Journal of intelligent & robotic systems 2022-05, Vol.105 (1), Article 12
• Main Authors: Muzio, Alexandre F. V., Maximo, Marcos R. O. A., Yoneyama, Takashi
• Format: Article
• Language: English
• Subjects: Agents (artificial intelligence); Algorithms; Artificial Intelligence; Control; Control tasks; Deep learning; Electrical Engineering; Engineering; Humanoid; Machine learning; Mechanical Engineering; Mechatronics; Racing; Regular Paper; Robotics; Robots; Simulation; Simulator fidelity; Soccer; Topical collection on LARS 2020
• ISSN: 0921-0296; 1573-0409
• DOI: 10.1007/s10846-022-01619-y

RoboCup 3D Soccer Simulation is a robot soccer competition based on a high-fidelity simulator with autonomous humanoid agents, making it an interesting testbed for robotics and artificial intelligence. Due to the recent success of Deep Reinforcement Learning (DRL) in continuous control tasks, many teams have been using this technique to develop motions in Soccer 3D. This article focuses on learning humanoid robot behaviors: completing a racing track as fast as possible and dribbling against a single opponent. Our approach uses a hierarchical controller where a model-free policy learns to interact model-based walking algorithm. Then, we use DRL algorithms for an agent to learn how to perform these behaviors. Finally, the learned dribble policy was evaluated in the Soccer 3D environment. Simulated experiments show that the DRL agent wins against the hand-coded behavior used by the ITAndroids robotics team in 68.2% of dribble attempts.