Online Learning with Hidden Markov Models

We present an online version of the expectation-maximization (EM) algorithm for hidden Markov models (HMMs). The sufficient statistics required for parameters estimation is computed recursively with time, that is, in an online way instead of using the batch forward-backward procedure. This computati...

Full description

Saved in:
Bibliographic Details
Published in:Neural computation 2008-07, Vol.20 (7), p.1706-1716
Main Authors: Mongillo, Gianluigi, Deneve, Sophie
Format: Article
Language:English
Subjects:
Algorithms
Applied sciences
Artificial Intelligence
Biological and medical sciences
Computer science
control theory
systems
Exact sciences and technology
Fundamental and applied biological sciences. Psychology
General aspects
Learning and adaptive systems
Markov analysis
Markov Chains
Mathematical models
Mathematics
Mathematics in biology. Statistical analysis. Models. Metrology. Data processing in biology (general aspects)
Miscellaneous
Neurosciences
Online instruction
Parameter estimation
Parametric inference
Probability
Probability and statistics
Sciences and techniques of general use
Statistics
Time Factors
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We present an online version of the expectation-maximization (EM) algorithm for hidden Markov models (HMMs). The sufficient statistics required for parameters estimation is computed recursively with time, that is, in an online way instead of using the batch forward-backward procedure. This computational scheme is generalized to the case where the model parameters can change with time by introducing a discount factor into the recurrence relations. The resulting algorithm is equivalent to the batch EM algorithm, for appropriate discount factor and scheduling of parameters update. On the other hand, the online algorithm is able to deal with dynamic environments, i.e., when the statistics of the observed data is changing with time. The implications of the online algorithm for probabilistic modeling in neuroscience are briefly discussed.
ISSN:0899-7667
1530-888X
DOI:10.1162/neco.2008.10-06-351