Mixtures of inverse covariances

• Published in: IEEE transactions on speech and audio processing 2004-05, Vol.12 (3), p.250-264
• Main Authors: Vanhoucke, V., Sankar, A.
• Format: Article
• Language: English
• Subjects: Algorithms; Applied sciences; Covariance; Covariance matrix; Density; Detection, estimation, filtering, equalization, prediction; Exact sciences and technology; Gaussian; Gaussian processes; Information, signal and communications theory; Inverse; Inverse problems; Mathematical models; Matrices; Maximum likelihood estimation; Parameter estimation; Prototypes; Real time systems; Robustness; Scalability; Signal and communications theory; Signal processing; Signal, noise; Speech processing; Speech recognition; Telecommunications and information theory
• ISSN: 1063-6676; 2329-9290; 1558-2353; 2329-9304
• DOI: 10.1109/TSA.2004.825675

We describe a model which approximates full covariances in a Gaussian mixture while reducing significantly both the number of parameters to estimate and the computations required to evaluate the Gaussian likelihoods. In this model, the inverse covariance of each Gaussian in the mixture is expressed as a linear combination of a small set of prototype matrices that are shared across components. In addition, we demonstrate the benefits of a subspace-factored extension of this model when representing independent or near-independent product densities. We present a maximum likelihood estimation algorithm for these models, as well as a practical method for implementing it. We show through experiments performed on a variety of speech recognition tasks that this model significantly outperforms a diagonal covariance model, while using far fewer Gaussian-specific parameters. Experiments also demonstrate that a better speed/accuracy tradeoff can be achieved on a real-time speech recognition system.