Finite-Time Adaptive Dynamic Programming for Affine-Form Nonlinear Systems

• Published in: IEEE transaction on neural networks and learning systems 2023-12, Vol.PP, p.1-14
• Main Authors: Zhang, Longjie, Chen, Yong
• Format: Article
• Language: English
• Subjects: Artificial neural networks; Asymptotic stability; Circuit stability; Control systems; Critic-actor algorithm; finite-time adaptive dynamic programming (FTADP); nonlinear systems; Optimal control; Optimization; reinforcement learning (RL); Stability criteria
• ISSN: 2162-237X; 2162-2388
• DOI: 10.1109/TNNLS.2023.3337387

Inspired by the fusion of state optimization and finite-time convergence, the finite-time optimal control (FTOC) for the affine-form nonlinear systems is investigated in this article. To achieve optimal stability with finite response time, a novel finite-time adaptive dynamic programming (FTADP) is presented for the affine-form nonlinear systems. By mapping the value function into finite-time stability space with the transformation function, the Bellman equation with finite-time stability space is first obtained. Then, by solving the Hamilton-Jacobi-Bellman (HJB) equation, the new FTOC strategy is presented with the theoretical finite-time stability description. Furthermore, to solve the above optimal controller with nonlinearity characteristic, the novel adaptive dynamic programming (ADP) based on the finite-time critic-actor offline neural network (NN) approximation algorithm is implemented, and the corresponding finite-time convergence characteristic is illustrated theoretically. Eventually, the application analysis on the circuit systems shows that the proposed FTADP has superiority compared with general optimal control.