A Direct Estimation Approach to Sparse Linear Discriminant Analysis

This article considers sparse linear discriminant analysis of high-dimensional data. In contrast to the existing methods which are based on separate estimation of the precision matrix Ω and the difference δ of the mean vectors, we introduce a simple and effective classifier by estimating the product...

Full description

Saved in:
Bibliographic Details
Published in:Journal of the American Statistical Association 2011-12, Vol.106 (496), p.1566-1577
Main Authors: Cai, Tony, Liu, Weidong
Format: Article
Language:English
Subjects:
Analytical estimating
Applications
Asymptotic methods
Cancer
Classification
Computational methods
Constrained l
Covariance matrices
Datasets
Discriminant analysis
Error rates
Estimation
Estimation methods
Exact sciences and technology
Fisher's rule
General topics
Leukemia
Linear discriminant analysis
Linear inference, regression
Linear models
Linear programming
Lung neoplasms
Mathematics
minimization
Multivariate analysis
Naive Bayes rule
Oracles
Probability and statistics
Sciences and techniques of general use
Sparsity
Statistical analysis
Statistics
Theory and Methods
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This article considers sparse linear discriminant analysis of high-dimensional data. In contrast to the existing methods which are based on separate estimation of the precision matrix Ω and the difference δ of the mean vectors, we introduce a simple and effective classifier by estimating the product Ωδ directly through constrained ℓ 1 minimization. The estimator can be implemented efficiently using linear programming and the resulting classifier is called the linear programming discriminant (LPD) rule. The LPD rule is shown to have desirable theoretical and numerical properties. It exploits the approximate sparsity of Ωδ and as a consequence allows cases where it can still perform well even when Ω and/or δ cannot be estimated consistently. Asymptotic properties of the LPD rule are investigated and consistency and rate of convergence results are given. The LPD classifier has superior finite sample performance and significant computational advantages over the existing methods that require separate estimation of Ω and δ. The LPD rule is also applied to analyze real datasets from lung cancer and leukemia studies. The classifier performs favorably in comparison to existing methods.
ISSN:0162-1459
1537-274X
DOI:10.1198/jasa.2011.tm11199