Information-Maximization Clustering Based on Squared-Loss Mutual Information

Information-maximization clustering learns a probabilistic classifier in an unsupervised manner so that mutual information between feature vectors and cluster assignments is maximized. A notable advantage of this approach is that it involves only continuous optimization of model parameters, which is...

Full description

Saved in:
Bibliographic Details
Published in:Neural computation 2014-01, Vol.26 (1), p.84-131
Main Authors: Sugiyama, Masashi, Niu, Gang, Yamada, Makoto, Kimura, Manabu, Hachiya, Hirotaka
Format: Article
Language:English
Subjects:
Algorithms
Cluster Analysis
Experiments
Face
Humans
Image Interpretation, Computer-Assisted - methods
Information management
Letters
Multivariate analysis
Optimization
Pattern Recognition, Automated - methods
Probability
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Information-maximization clustering learns a probabilistic classifier in an unsupervised manner so that mutual information between feature vectors and cluster assignments is maximized. A notable advantage of this approach is that it involves only continuous optimization of model parameters, which is substantially simpler than discrete optimization of cluster assignments. However, existing methods still involve nonconvex optimization problems, and therefore finding a good local optimal solution is not straightforward in practice. In this letter, we propose an alternative information-maximization clustering method based on a squared-loss variant of mutual information. This novel approach gives a clustering solution analytically in a computationally efficient way via kernel eigenvalue decomposition. Furthermore, we provide a practical model selection procedure that allows us to objectively optimize tuning parameters included in the kernel function. Through experiments, we demonstrate the usefulness of the proposed approach.
ISSN:0899-7667
1530-888X
DOI:10.1162/NECO_a_00534