The Bethe approximation of the pattern maximum likelihood distribution

• Main Author: Vontobel, P. O.
• Format: Conference Proceeding
• Language: English
• Subjects: Approximation algorithms; Approximation methods; Graphical models; Maximum likelihood estimation; Mercury (metals); Minimization; Vectors
• ISSN: 2157-8095; 2157-8117
• DOI: 10.1109/ISIT.2012.6283654

Among all memoryless source distributions, the pattern maximum likelihood (PML) distribution is the distribution which maximizes the probability that a memoryless source produces a string with a given pattern. Equivalently, the PML distribution maximizes the permanent of a certain non-negative matrix. We reformulate this maximization problem as a double minimization problem of a suitable Gibbs free energy function. Because finding the minimum of this function appears intractable for practically relevant problem sizes, one must look for tractable approximations. One approach is to approximately find a minimum (or at least a local minimum) of the Gibbs free energy function by applying an alternating minimization algorithm where the steps are based on quantities that are obtained by Markov chain Monte Carlo sampling. One can show that this approach is equivalent to an algorithm that was proposed by Orlitsky et al. An alternative approach is to replace the Gibbs free energy function by a tractable approximation like the Bethe free energy function and to apply an alternating minimization algorithm to this function. As it turns out, empirically, this latter approach gives very good approximations to the PML distribution (or at least a locally optimal PML distribution), and, for the same level of accuracy, is two to three orders of magnitude faster than the former approach for practically relevant problem sizes. Moreover, the above free energy framework allows us to simplify some earlier proofs of properties of the PML distribution and to derive some new properties of the PML distribution, along with obtaining similar results for its Bethe approximation.