A countably infinite mixture model for clustering and feature selection

Mixture modeling is one of the most useful tools in machine learning and data mining applications. An important challenge when applying finite mixture models is the selection of the number of clusters which best describes the data. Recent developments have shown that this problem can be handled by t...

Full description

Saved in:
Bibliographic Details
Published in:Knowledge and information systems 2012-11, Vol.33 (2), p.351-370
Main Authors: Bouguila, Nizar, Ziou, Djemel
Format: Article
Language:English
Subjects:
Algorithms
Applied sciences
Artificial intelligence
Bayesian analysis
Cluster analysis
Clustering
Computer Science
Computer science
control theory
systems
Computer simulation
Data mining
Data Mining and Knowledge Discovery
Data processing. List processing. Character string processing
Database Management
Dirichlet problem
Exact sciences and technology
Feature selection
Information Storage and Retrieval
Information Systems and Communication Service
Information Systems Applications (incl.Internet)
IT in Business
Mathematical analysis
Mathematical models
Memory organisation. Data processing
Monte Carlo methods
Pattern recognition. Digital image processing. Computational geometry
Probability
Regular Paper
Software
Speech and sound recognition and synthesis. Linguistics
Studies
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Mixture modeling is one of the most useful tools in machine learning and data mining applications. An important challenge when applying finite mixture models is the selection of the number of clusters which best describes the data. Recent developments have shown that this problem can be handled by the application of non-parametric Bayesian techniques to mixture modeling. Another important crucial preprocessing step to mixture learning is the selection of the most relevant features. The main approach in this paper, to tackle these problems, consists on storing the knowledge in a generalized Dirichlet mixture model by applying non-parametric Bayesian estimation and inference techniques. Specifically, we extend finite generalized Dirichlet mixture models to the infinite case in which the number of components and relevant features do not need to be known a priori. This extension provides a natural representation of uncertainty regarding the challenging problem of model selection. We propose a Markov Chain Monte Carlo algorithm to learn the resulted infinite mixture. Through applications involving text and image categorization, we show that infinite mixture models offer a more powerful and robust performance than classic finite mixtures for both clustering and feature selection.
ISSN:0219-1377
0219-3116
DOI:10.1007/s10115-011-0467-4