Efficient and Safe Exploration in Deterministic Markov Decision Processes with Unknown Transition Models

We propose a safe exploration algorithm for deterministic Markov Decision Processes with unknown transition models. Our algorithm guarantees safety by leveraging Lipschitz-continuity to ensure that no unsafe states are visited during exploration. Unlike many other existing techniques, the provided s...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2019-04
Main Authors: Erdem Bıyık, Margoliash, Jonathan, Shahrouz Ryan Alimo, Sadigh, Dorsa
Format: Article
Language:English
Subjects:
Algorithms
Computer simulation
Exploration
Markov analysis
Markov processes
Safety
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We propose a safe exploration algorithm for deterministic Markov Decision Processes with unknown transition models. Our algorithm guarantees safety by leveraging Lipschitz-continuity to ensure that no unsafe states are visited during exploration. Unlike many other existing techniques, the provided safety guarantee is deterministic. Our algorithm is optimized to reduce the number of actions needed for exploring the safe space. We demonstrate the performance of our algorithm in comparison with baseline methods in simulation on navigation tasks.
ISSN:2331-8422
DOI:10.48550/arxiv.1904.01068