Links between Markov models and multilayer perceptrons

• Published in: IEEE transactions on pattern analysis and machine intelligence 1990-12, Vol.12 (12), p.1167-1178
• Main Authors: Bourlard, H., Wellekens, C.J.
• Format: Article
• Language: English
• Subjects: Applied sciences; Artificial intelligence; Computer science; control theory; systems; Context modeling; Entropy; Exact sciences and technology; Feedback; Hidden Markov models; Least squares approximation; Minimization methods; Multilayer perceptrons; Probability distribution; Speech and sound recognition and synthesis. Linguistics; Speech recognition; Viterbi algorithm
• ISSN: 0162-8828; 1939-3539
• DOI: 10.1109/34.62605

The statistical use of a particular classic form of a connectionist system, the multilayer perceptron (MLP), is described in the context of the recognition of continuous speech. A discriminant hidden Markov model (HMM) is defined, and it is shown how a particular MLP with contextual and extra feedback input units can be considered as a general form of such a Markov model. A link between these discriminant HMMs, trained along the Viterbi algorithm, and any other approach based on least mean square minimization of an error function (LMSE) is established. It is shown theoretically and experimentally that the outputs of the MLP (when trained along the LMSE or the entropy criterion) approximate the probability distribution over output classes conditioned on the input, i.e. the maximum a posteriori probabilities. Results of a series of speech recognition experiments are reported. The possibility of embedding MLP into HMM is described. Relations with other recurrent networks are also explained.< >