Refining hidden Markov models with recurrent neural networks

Both hidden Markov models (HMMs) and recurrent neural networks (RNNs) have been applied to sequence recognition problems. While HMMs are easy to train, they generally do not perform satisfactorily on difficult recognition problems. On the other hand, RNNs are excellent recognizers but are very hard...

Full description

Saved in:
Bibliographic Details
Main Authors: Wessels, T., Omlin, C.W.
Format: Conference Proceeding
Language:English
Subjects:
Computer science
Data mining
Encoding
Hidden Markov models
Neural networks
Optimization methods
Pattern classification
Recurrent neural networks
Simulated annealing
Training data
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Both hidden Markov models (HMMs) and recurrent neural networks (RNNs) have been applied to sequence recognition problems. While HMMs are easy to train, they generally do not perform satisfactorily on difficult recognition problems. On the other hand, RNNs are excellent recognizers but are very hard to train. Hybrid HMM/NN approaches aim at taking advantage of the strengths of both paradigms while avoiding their respective weaknesses. The paper proposes an approach of combining HMMs with RNNs. We discuss an algorithm for directly mapping a trained HMM into a RNN architecture and derive a gradient-descent learning algorithm for knowledge refinement.
ISSN:1098-7576
1558-3902
DOI:10.1109/IJCNN.2000.857908