A STEPWISE REGRESSION METHOD AND CONSISTENT MODEL SELECTION FOR HIGH-DIMENSIONAL SPARSE LINEAR MODELS

We introduce a fast stepwise regression method, called the orthogonal greedy algorithm (OGA), that selects input variables to enter a p-dimensional linear regression model (with p ≫ n, the sample size) sequentially so that the selected variable at each step minimizes the residual sum squares. We der...

Full description

Saved in:
Bibliographic Details
Published in:Statistica Sinica 2011-10, Vol.21 (4), p.1473-1513
Main Authors: Ing, Ching-Kang, Lai, Tze Leung
Format: Article
Language:English
Subjects:
Approximation
Covariance matrices
Greedy algorithms
Least squares
Linear regression
Modeling
Oracles
Regression analysis
Regression coefficients
Simulations
Citations: Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We introduce a fast stepwise regression method, called the orthogonal greedy algorithm (OGA), that selects input variables to enter a p-dimensional linear regression model (with p ≫ n, the sample size) sequentially so that the selected variable at each step minimizes the residual sum squares. We derive the convergence rate of OGA and develop a consistent model selection procedure along the OGA path that can adjust for potential spuriousness of the greedily chosen regressors among a large number of candidate variables. The resultant regression estimate is shown to have the oracle property of being equivalent to least squares regression on an asymptotically minimal set of relevant regressors under a strong sparsity condition.
ISSN:1017-0405
1996-8507
DOI:10.5705/ss.2010.081