Value iteration with deep neural networks for optimal control of input-affine nonlinear systems

• Published in: SICE Journal of Control, Measurement, and System Integration Measurement, and System Integration, 2021-01, Vol.14 (1), p.140-149
• Main Authors: Beppu, Hirofumi, Maruta, Ichiro, Fujimoto, Kenji
• Format: Article
• Language: English
• Subjects: convergence analysis; deep neural networks; input-affine nonlinear systems; optimal control; Value iteration
• ISSN: 1882-4889; 1884-9970
• DOI: 10.1080/18824889.2021.1936817

This paper proposes a new algorithm with deep neural networks to solve optimal control problems for continuous-time input nonlinear systems based on a value iteration algorithm. The proposed algorithm applies the networks to approximating the value functions and control inputs in the iterations. Consequently, the partial differential equations of the original algorithm reduce to the optimization problems for the parameters of the networks. Although the conventional algorithm can obtain the optimal control with iterative computations, each of the computations needs to be completed precisely, and it is hard to achieve sufficient precision in practice. Instead, the proposed method provides a practical method using deep neural networks and overcomes the difficulty based on a property of the networks, under which our convergence analysis shows that the proposed algorithm can achieve the minimum of the value function and the corresponding optimal controller. The effectiveness of the proposed method even with reasonable computational resources is demonstrated in two numerical simulations.