Input-Decoupled Q-Learning for Optimal Control

A design of optimal controllers based on a reinforcement learning method called Q-Learning is presented. Central to Q-Learning is the Q-function which is a function of the state and all input variables. This paper shows that decoupled-in-the-inputs Q-functions exist, and can be used to find the opti...

Full description

Saved in:
Bibliographic Details
Published in:The Journal of the astronautical sciences 2020-06, Vol.67 (2), p.630-656
Main Authors: Phan, Minh Q., Azad, Seyed Mahdi B.
Format: Article
Language:English
Subjects:
Aerospace Technology and Astronautics
Control theory
Controllers
Engineering
Linear systems
Machine learning
Mathematical Applications in the Physical Sciences
Nonlinear systems
Optimal control
Optimization
Space Exploration and Astronautics
Space Sciences (including Extraterrestrial Physics
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:A design of optimal controllers based on a reinforcement learning method called Q-Learning is presented. Central to Q-Learning is the Q-function which is a function of the state and all input variables. This paper shows that decoupled-in-the-inputs Q-functions exist, and can be used to find the optimal controllers for each input individually. The method thus converts a multiple-variable optimization problem into much simpler single-variable optimization problems while achieving optimality. An explicit model of the system is not required to learn these decoupled Q-functions, but rather the method relies on the ability to probe the system and observe its state transition. Derived within the framework of modern control theory, the method is applicable to both linear and non-linear systems.
ISSN:0021-9142
2195-0571
DOI:10.1007/s40295-019-00157-4