A Variance Minimization Criterion to Feature Selection Using Laplacian Regularization

In many information processing tasks, one is often confronted with very high-dimensional data. Feature selection techniques are designed to find the meaningful feature subset of the original features which can facilitate clustering, classification, and retrieval. In this paper, we consider the featu...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on pattern analysis and machine intelligence 2011-10, Vol.33 (10), p.2013-2025
Main Authors: He, Xiaofei, Ji, Ming, Zhang, Chiyuan, Bao, Hujun
Format: Article
Language:English
Subjects:
Algorithm design and analysis
Applied sciences
clustering
Clustering algorithms
Computer science
control theory
systems
Covariance matrix
Data processing. List processing. Character string processing
dimensionality reduction
Economic models
Exact sciences and technology
Feature extraction
Feature selection
Laplace equations
manifold
Manifolds
Memory organisation. Data processing
Optimization
regression
regularization
Software
Studies
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In many information processing tasks, one is often confronted with very high-dimensional data. Feature selection techniques are designed to find the meaningful feature subset of the original features which can facilitate clustering, classification, and retrieval. In this paper, we consider the feature selection problem in unsupervised learning scenarios, which is particularly difficult due to the absence of class labels that would guide the search for relevant information. Based on Laplacian regularized least squares, which finds a smooth function on the data manifold and minimizes the empirical loss, we propose two novel feature selection algorithms which aim to minimize the expected prediction error of the regularized regression model. Specifically, we select those features such that the size of the parameter covariance matrix of the regularized regression model is minimized. Motivated from experimental design, we use trace and determinant operators to measure the size of the covariance matrix. Efficient computational schemes are also introduced to solve the corresponding optimization problems. Extensive experimental results over various real-life data sets have demonstrated the superiority of the proposed algorithms.
ISSN:0162-8828
1939-3539
2160-9292
DOI:10.1109/TPAMI.2011.44