Terrain Adaptive Walking of Biped Neuromuscular Virtual Human Using Deep Reinforcement Learning

There have been some biomechanics-based control systems that have achieved better realistic virtual human motion. Yet their abilities to adapt the changing environments are weaker than the traditional control systems with characters driven by proportional derivative actuators directly. In our method...

Full description

Saved in:
Bibliographic Details
Published in:IEEE access 2019, Vol.7, p.92465-92475
Main Authors: Wang, Jianpeng, Qin, Wenhu, Sun, Libo
Format: Article
Language:English
Subjects:
Actuators
Adaptive control
Algorithms
Artificial neural networks
Biological system modeling
Biomechanics
Biped walking
Changing environments
Feedback
Force
hierarchical control
Human motion
Human performance
Legged locomotion
Machine learning
Motion control
Muscles
Neural networks
Neuromuscular
neuromuscular virtual human
Optimization algorithms
Particle swarm optimization
Proportional derivative
proximal policy optimization
Solid modeling
Terrain
Virtual humans
Walking
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:There have been some biomechanics-based control systems that have achieved better realistic virtual human motion. Yet their abilities to adapt the changing environments are weaker than the traditional control systems with characters driven by proportional derivative actuators directly. In our method, we build a hierarchical neuromuscular virtual human (NMVH) motion control system that consists of a low-level spine reflex layer and a high-level policy control layer. The spine reflex layer uses a feedback net to map sensory information to excitations, which stimulate muscles to generate joint torques. The policy control layer includes a deep neural network, which provides a learned action policy to spine reflex layer for achieving terrain-adaptive motion skills. The particle swarm optimization algorithm is used to optimize the gain factors of the feedback net for finding out a basic policy to make the virtual human walk on the flat terrain autonomously. The proximal policy optimization algorithm is employed to train the deep neural network in policy control layer for learning how to modulate the actions to adapt to the changing terrain. The simulation results in Matlab show that virtual human can walk smoothly and better adapt to the given terrain changes. It demonstrates that our control system improves the terrain-adaptive walking skill of the neuromuscular virtual human.
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2019.2927606