Perturbation analysis of inhomogeneous finite Markov chains

In this paper we provide a perturbation analysis of finite time-inhomogeneous Markov processes. We derive closed-form representations for the derivative of the transition probability at time t, with t > 0. Elaborating on this result, we derive simple gradient estimators for transient performance...

Full description

Saved in:
Bibliographic Details
Published in:Advances in applied probability 2016-03, Vol.48 (1), p.255-273
Main Authors: Heidergott, Bernd, Leahu, Haralambie, Löpker, Andreas, Pflug, Georg
Format: Article
Language:English
Subjects:
60J27
65C05
Analytical estimating
Derivatives
Ergodic theory
Estimating techniques
Estimators
Exact solutions
Finite Markov process
infinitesimal generator
inhomogeneous Markov process
Markov analysis
Markov chains
Markov processes
Mathematical analysis
Matrix
Perturbation methods
Poisson process
Probability
Retrials
Sensitivity analysis
Simplification
Staffing
Studies
Transient performance
Transition probabilities
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this paper we provide a perturbation analysis of finite time-inhomogeneous Markov processes. We derive closed-form representations for the derivative of the transition probability at time t, with t > 0. Elaborating on this result, we derive simple gradient estimators for transient performance characteristics either taken at some fixed point in time t, or for the integrated performance over a time interval [0 , t]. Bounds for transient performance sensitivities are presented as well. Eventually, we identify a structural property of the derivative of the generator matrix of a Markov chain that leads to a significant simplification of the estimators.
ISSN:0001-8678
1475-6064
DOI:10.1017/apr.2015.16