Transferring Online Reinforcement Learning for Electric Motor Control From Simulation to Real-World Experiments

Reinforcement learning (RL) based methods are an upcoming approach for the control of power systems such as electric drives. These data-driven techniques do not need an explicit plant model like most common state-of-the-art approaches. Instead, the control policy is continuously improved solely base...

Full description

Saved in:
Bibliographic Details
Published in:IEEE open journal of power electronics 2021, Vol.2, p.187-201
Main Authors: Book, Gerrit, Traue, Arne, Balakrishna, Praneeth, Brosch, Anian, Schenke, Maximilian, Hanke, Soren, Kirchgassner, Wilhelm, Wallscheid, Oliver
Format: Article
Language:English
Subjects:
Delays
Edge computing
electric drive
machine learning
Mathematical model
optimal control
Power electronics
Real-time systems
reinforcement learning
Safety
Stators
Training
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Reinforcement learning (RL) based methods are an upcoming approach for the control of power systems such as electric drives. These data-driven techniques do not need an explicit plant model like most common state-of-the-art approaches. Instead, the control policy is continuously improved solely based on measurement feedback pursuing optimal control performance through learning. While the general feasibility of RL-based drive control algorithms has already been proven in simulation, this work focuses on transferring the methodology to real-world experiments. In the case of electric motor control, a strict real-time requirement, safety constraints, system delays and the limitations of embedded hardware frameworks are hurdles to overcome. Hence, several modifications to the general RL training setup are introduced in order to enable RL in real-world electric drive control problems. In particular, a rapid control prototyping toolchain is introduced allowing fast and flexible testing of arbitrary RL algorithms. This simulation-to-experiment pipeline is considered an important intermediate step towards introducing RL in embedded control for power electronic systems. To highlight the potential of RL-based drive control, extensive experimental investigations addressing the current control of a permanent magnet synchronous motor utilizing a deep deterministic policy gradient algorithm have been conducted. Despite the early state of research in this domain, promising control performance could be achieved.
ISSN:2644-1314
2644-1314
DOI:10.1109/OJPEL.2021.3065877