Development of a New Robust Stable Walking Algorithm for a Humanoid Robot Using Deep Reinforcement Learning with Multi-Sensor Data Fusion

The difficult task of creating reliable mobility for humanoid robots has been studied for decades. Even though several different walking strategies have been put forth and walking performance has substantially increased, stability still needs to catch up to expectations. Applications for Reinforceme...

Full description

Saved in:
Bibliographic Details
Published in:Electronics (Basel) 2023-01, Vol.12 (3), p.568
Main Authors: Kaymak, Çağrı, Uçar, Ayşegül, Güzeliş, Cüneyt
Format: Article
Language:English
Subjects:
Algorithms
Artificial intelligence
Balancing
Control theory
Data integration
Deep learning
Design
Design and construction
Designers
Gait
Humanoid
Locomotion
Machine learning
Mobile robots
Motion
Multisensor fusion
Optimization
Parameters
Reinforcement learning (Machine learning)
Robot dynamics
Robots
Robustness
Sensors
Trajectory control
Walking
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The difficult task of creating reliable mobility for humanoid robots has been studied for decades. Even though several different walking strategies have been put forth and walking performance has substantially increased, stability still needs to catch up to expectations. Applications for Reinforcement Learning (RL) techniques are constrained by low convergence and ineffective training. This paper develops a new robust and efficient framework based on the Robotis-OP2 humanoid robot combined with a typical trajectory-generating controller and Deep Reinforcement Learning (DRL) to overcome these limitations. This framework consists of optimizing the walking trajectory parameters and posture balancing system. Multi-sensors of the robot are used for parameter optimization. Walking parameters are optimized using the Dueling Double Deep Q Network (D3QN), one of the DRL algorithms, in the Webots simulator. The hip strategy is adopted for the posture balancing system. Experimental studies are carried out in both simulation and real environments with the proposed framework and Robotis-OP2’s walking algorithm. Experimental results show that the robot performs more stable walking with the proposed framework than Robotis-OP2’s walking algorithm. It is thought that the proposed framework will be beneficial for researchers studying in the field of humanoid robot locomotion.
ISSN:2079-9292
2079-9292
DOI:10.3390/electronics12030568