Controllability and decentralized stabilization of the Kolmogorov forward equation for Markov chains

• Published in: Automatica (Oxford) 2021-02, Vol.124, p.109351, Article 109351
• Main Authors: Elamvazhuthi, Karthik, Biswal, Shiba, Berman, Spring
• Format: Article
• Language: English
• Subjects: Autonomous mobile robots; Bilinear control systems; Continuous-time Markov chains; Controllability; Swarm robotics
• ISSN: 0005-1098
• DOI: 10.1016/j.automatica.2020.109351

In this paper, we provide several results on controllability and stabilizability properties of the Kolmogorov forward equation of a continuous-time Markov chain (CTMC) evolving on a finite state space, with the transition rates defined as the control parameters. First, we show that any target probability distribution can be reached asymptotically using time-varying control parameters. Second, we characterize all stationary distributions that are stabilizable using time-independent control parameters. For bidirected graphs, we construct rational and polynomial density feedback laws that stabilize stationary distributions while satisfying the additional constraint that the feedback law takes zero value at equilibrium. This last result enables the construction of decentralized density feedback controllers, using tools from linear systems theory and sum-of-squares based polynomial optimization, that stabilize a swarm of robots modeled as a CTMC to a target state distribution with no state-switching at equilibrium. In addition to these results, we prove a sufficient condition under which the classical rank conditions for controllability can be generalized to forward equations with non-negativity constraints on the control inputs. We apply this result to prove local controllability of a forward equation in which only a small subset of the transition rates are the control inputs. Lastly, we extend our feedback stabilization results to stationary distributions that have a strongly connected support.