Manipulator Trajectory Optimization Using Reinforcement Learning on a Reduced-Order Dynamic Model with Deep Neural Network Compensation

• Published in: Machines (Basel) 2023-03, Vol.11 (3), p.350
• Main Authors: Chen, Yung-Hsiu, Yang, Wu-Te, Chen, Bo-Hsun, Lin, Pei-Chun
• Format: Article
• Language: English
• Subjects: Artificial intelligence; Artificial neural networks; Cameras; Damping; Deep learning; deep reinforcement learning; Digital twins; Dynamic models; Dynamic programming; Energy consumption; Energy efficiency; energy/speed optimization; Factories; Genetic algorithms; Machine learning; Model reduction; Neural networks; Obstacle avoidance; Optimization; Planning; Reduced order models; Robots; Trajectory optimization; Trajectory planning
• ISSN: 2075-1702; 2075-1702
• DOI: 10.3390/machines11030350

This article reports the construction of an articulated manipulator’s hybrid dynamic model and trajectory planning and optimization of the manipulator using deep reinforcement learning (RL) on the dynamic model. The hybrid model was composed of a physical-based reduced-order dynamic model, linear friction and damping terms, and a deep neural network model to compensate for the nonlinear characteristics of the manipulator. The hybrid model then served as the digital twin of the manipulator for trajectory planning to optimize energy efficiency and operation speed by using RL while taking obstacle avoidance into consideration. The proposed strategy was simulated and experimentally validated. The energy consumption along paths was reduced and the speed was increased so the manipulator could achieve more efficient motion.