Successive over relaxation for model-free LQR control of discrete-time Markov jump systems

This paper aims to solve the model-free linear quadratic regulator problem for discrete-time Markov jump linear systems without requiring an initial stabilizing control policy. We propose both model-based and model-free successive over relaxation algorithms to learn the optimal control policy of dis...

Full description

Saved in:
Bibliographic Details
Published in:Automatica (Oxford) 2025-01, Vol.171, p.111919, Article 111919
Main Authors: Fan, Wenwu, Xiong, Junlin
Format: Article
Language:English
Subjects:
Linear quadratic regulator
Markov jump linear systems
Model-free
Successive over relaxation
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper aims to solve the model-free linear quadratic regulator problem for discrete-time Markov jump linear systems without requiring an initial stabilizing control policy. We propose both model-based and model-free successive over relaxation algorithms to learn the optimal control policy of discrete-time Markov jump linear systems. The model-free value iteration algorithm is a special case of our model-free algorithm when the relaxation factor equals one. A sufficient condition on the relaxation factor is provided to guarantee the convergence of our algorithms. Moreover, it is proved that our model-free algorithm can obtain an approximate optimal solution when the transition probability matrix is unknown. Finally, a numerical example is used to illustrate our results.
ISSN:0005-1098
DOI:10.1016/j.automatica.2024.111919