Strong Uniform Value in Gambling Houses and Partially Observable Markov Decision Processes

In several standard models of dynamic programming (gambling houses, MDPs, POMDPs), we prove the existence of a robust notion of value for the infinitely repeated problem, namely the strong uniform value. This solves two open problems. First, this shows that for any > 0, the decision-maker has a p...

Full description

Saved in:
Bibliographic Details
Published in:SIAM journal on control and optimization 2016-01, Vol.54 (4), p.1983-2008
Main Authors: Venel, Xavier, Ziliotto, Bruno
Format: Article
Language:English
Subjects:
Economics and Finance
Humanities and Social Sciences
Mathematics
Optimization and Control
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In several standard models of dynamic programming (gambling houses, MDPs, POMDPs), we prove the existence of a robust notion of value for the infinitely repeated problem, namely the strong uniform value. This solves two open problems. First, this shows that for any > 0, the decision-maker has a pure strategy σ which is-optimal in any n-stage problem, provided that n is big enough (this result was only known for behavior strategies, that is, strategies which use randomization). Second, for any > 0, the decision-maker can guarantee the limit of the n-stage value minus in the infinite problem where the payoff is the expectation of the inferior limit of the time average payoff.
ISSN:0363-0129
1095-7138
DOI:10.1137/15M1043340