A Hierarchical Bayesian Model for Learning Nonlinear Statistical Regularities in Nonstationary Natural Signals

Capturing statistical regularities in complex, high-dimensional data is an important problem in machine learning and signal processing. Models such as principal component analysis (PCA) and independent component analysis (ICA) make few assumptions about the structure in the data and have good scalin...

Full description

Saved in:
Bibliographic Details
Published in:Neural computation 2005-02, Vol.17 (2), p.397-423
Main Authors: Karklin, Yan, Lewicki, Michael S.
Format: Article
Language:English
Subjects:
Applied sciences
Artificial intelligence
Bayes Theorem
Computer science
control theory
systems
Exact sciences and technology
Learning
Learning and adaptive systems
Mathematics
Multivariate analysis
Nonlinear Dynamics
Probability and statistics
Probability theory and stochastic processes
Sciences and techniques of general use
Special processes (renewal theory, markov renewal processes, semi-markov processes, statistical mechanics type models, applications)
Statistics
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Capturing statistical regularities in complex, high-dimensional data is an important problem in machine learning and signal processing. Models such as principal component analysis (PCA) and independent component analysis (ICA) make few assumptions about the structure in the data and have good scaling properties, but they are limited to representing linear statistical regularities and assume that the distribution of the data is stationary. For many natural, complex signals, the latent variables often exhibit residual dependencies as well as nonstationary statistics. Here we present a hierarchical Bayesian model that is able to capture higher-order nonlinear structure and represent nonstationary data distributions. The model is a generalization of ICA in which the basis function coefficients are no longer assumed to be independent; instead, the dependencies in their magnitudes are captured by a set of density components. Each density component describes a common pattern of deviation from the marginal density of the pattern ensemble; in different combinations, they can describe nonstationary distributions. Adapting the model to image or audio data yields a nonlinear, distributed code for higher-order statistical regularities that reflect more abstract, invariant properties of the signal.
ISSN:0899-7667
1530-888X
DOI:10.1162/0899766053011474