Deep Neural Networks Algorithms for Stochastic Control Problems on Finite Horizon: Convergence Analysis

This paper develops algorithms for high-dimensional stochastic control problems based on deep learning and dynamic programming. Unlike classical approximate dynamic programming approaches, we first approximate the optimal policy by means of neural networks in the spirit of deep reinforcement learnin...

Full description

Saved in:
Bibliographic Details
Published in:SIAM journal on numerical analysis 2021-01, Vol.59 (1), p.525-557
Main Authors: Huré, Côme, Pham, Huyên, Bachouch, Achref, Langrené, Nicolas
Format: Article
Language:English
Subjects:
Machine Learning
Mathematics
Optimization and Control
Probability
Statistics
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper develops algorithms for high-dimensional stochastic control problems based on deep learning and dynamic programming. Unlike classical approximate dynamic programming approaches, we first approximate the optimal policy by means of neural networks in the spirit of deep reinforcement learning, and then the value function by Monte Carlo regression. This is achieved in the dynamic programming recursion by performance or hybrid iteration and regress-now methods from numerical probabilities. We provide a theoretical justification of these algorithms. Consistency and rate of convergence for the control and value function estimates are analyzed and expressed in terms of the universal approximation error of the neural networks, and of the statistical error when estimating network function, leaving aside the optimization error. Numerical results on various applications are presented in a companion paper [Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications, Methodol. Comput. Appl. Probab., 24(1) 143-178 2022] and illustrate the performance of the proposed algorithms.
ISSN:0036-1429
1095-7170
DOI:10.1137/20M1316640