A Novel Deep Density Model for Unsupervised Learning

Density models are fundamental in machine learning and have received a widespread application in practical cognitive modeling tasks and learning problems. In this work, we introduce a novel deep density model, referred to as deep mixtures of factor analyzers with common loadings (DMCFA), with an eff...

Full description

Saved in:
Bibliographic Details
Published in:Cognitive computation 2019-12, Vol.11 (6), p.778-788
Main Authors: Yang, Xi, Huang, Kaizhu, Zhang, Rui, Goulermas, John Y.
Format: Article
Language:English
Subjects:
Algorithms
Analyzers
Artificial Intelligence
Biomedical and Life Sciences
Biomedicine
Clustering
Cognitive tasks
Computation by Abstract Devices
Computational Biology/Bioinformatics
Deep learning
Density
Estimation
Feature selection
Greedy algorithms
Machine learning
Mixtures
Neurosciences
Unsupervised learning
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Density models are fundamental in machine learning and have received a widespread application in practical cognitive modeling tasks and learning problems. In this work, we introduce a novel deep density model, referred to as deep mixtures of factor analyzers with common loadings (DMCFA), with an efficient greedy layer-wise unsupervised learning algorithm. The model employs a mixture of factor analyzers sharing common component loadings in each layer. The common loadings can be considered to be a feature selection or reduction matrix which makes this new model more physically meaningful. Importantly, sharing common components is capable of reducing both the number of free parameters and computation complexity remarkably. Consequently, DMCFA makes inference and learning rely on a dramatically more succinct model and avoids sacrificing its flexibility in estimating the data density by utilizing Gaussian distributions as the priors. Our model is evaluated on five real datasets and compared to three other competitive models including mixtures of factor analyzers (MFA), MFA with common loadings (MCFA), deep mixtures of factor analyzers (DMFA), and their collapsed counterparts. The results demonstrate the superiority of the proposed model in the tasks of density estimation, clustering, and generation.
ISSN:1866-9956
1866-9964
DOI:10.1007/s12559-018-9566-9