Integral reinforcement learning off-policy method for solving nonlinear multi-player nonzero-sum games with saturated actuator

• Published in: Neurocomputing (Amsterdam) 2019-03, Vol.335, p.96-104
• Main Authors: Ren, He, Zhang, Huaguang, Wen, Yinlei, Liu, Chong
• Format: Article
• Language: English
• Subjects: Adaptive dynamic programming; Constrained optimal control; Nonzero-sum game; Off-policy; Reinforcement learning
• ISSN: 0925-2312; 1872-8286
• DOI: 10.1016/j.neucom.2019.01.033

In this paper, an effective off-policy algorithm is proposed to solve the continuous time nonzero-sum (NZS) control problem for unknown nonlinear systems with saturated actuator. A class of nonquadratic function is used to construct the performance functions to deal with constrained inputs. Utilizing the integral reinforcement learning (IRL) technique, the off-policy learning mechanism is introduced to design an iterative method for the continuous-time NZS constrained control problem without requiring the knowledge of system dynamics. To show the convergence of the proposed method, the traditional policy iteration (PI) method is discussed for the continuous-time NZS control problem with saturated actuator at first. Then, the equivalence of the proposed method with the traditional PI method is proved. Neural networks are introduced to construct the actor-critic structure, where the critic neural networks are aimed at approximating the iterative value functions and the actor neural networks are aimed at approximating the iterative control policies. Finally, two cases are simulated to verify the effectiveness of the proposed method.