A Minimum Discounted Reward Hamilton-Jacobi Formulation for Computing Reachable Sets

We propose a novel formulation for approximating reachable sets through a minimum discounted reward optimal control problem. The formulation yields a continuous solution that can be obtained by solving a Hamilton-Jacobi equation. Furthermore, the numerical approximation to this solution is the uniqu...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on automatic control 2024-02, Vol.69 (2), p.1-8
Main Authors: Akametalu, Anayo K., Ghosh, Shromona, Fisac, Jaime F., Rubies-Royo, Vicenc, Tomlin, Claire J.
Format: Article
Language:English
Subjects:
Algorithms
Approximation
Convergence
Double integrators
Fixed points (mathematics)
Games
Hamilton-Jacobi equation
Infinite horizon
Mathematical analysis
Optimal control
Pursuit-evasion games
Q measurement
Trajectory
Viscosity
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We propose a novel formulation for approximating reachable sets through a minimum discounted reward optimal control problem. The formulation yields a continuous solution that can be obtained by solving a Hamilton-Jacobi equation. Furthermore, the numerical approximation to this solution is the unique fixed-point to a contraction mapping. This allows for more efficient solution methods that are not applicable under traditional formulations for solving reachable sets. Lastly, this formulation provides a link between reinforcement learning and learning reachable sets for systems with unknown dynamics, allowing algorithms from the former to be applied to the latter. We use two benchmark examples, double integrator, and pursuit-evasion games, to show the correctness of the formulation as well as its strengths in comparison to previous work.
ISSN:0018-9286
1558-2523
DOI:10.1109/TAC.2023.3327159