Regularization Parameter Selections via Generalized Information Criterion

We apply the nonconcave penalized likelihood approach to obtain variable selections as well as shrinkage estimators. This approach relies heavily on the choice of regularization parameter, which controls the model complexity. In this paper, we propose employing the generalized information criterion,...

Full description

Saved in:
Bibliographic Details
Published in:Journal of the American Statistical Association 2010-03, Vol.105 (489), p.312-323
Main Authors: Zhang, Yiyun, Li, Runze, Tsai, Chih-Ling
Format: Article
Language:English
Subjects:
Akaike information criterion
Bayesian analysis
Bayesian information criterion
Criteria
Deadweight loss
Estimators
Generalized linear model
Least absolute shrinkage and selection operator
Least squares
Linear regression
Modeling
Nonconcave penalized likelihood
Oracles
Owls
Parameter identification
Parametric models
Regression analysis
Sample size
Simulation
Smoothly clipped absolute deviation
Statistics
Theory and Methods
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We apply the nonconcave penalized likelihood approach to obtain variable selections as well as shrinkage estimators. This approach relies heavily on the choice of regularization parameter, which controls the model complexity. In this paper, we propose employing the generalized information criterion, encompassing the commonly used Akaike information criterion (AIC) and Bayesian information criterion (BIC), for selecting the regularization parameter. Our proposal makes a connection between the classical variable selection criteria and the regularization parameter selections for the nonconcave penalized likelihood approaches. We show that the BIC-type selector enables identification of the true model consistently, and the resulting estimator possesses the oracle property in the terminology of Fan and Li (2001). In contrast, however, the AIC-type selector tends to overfit with positive probability. We further show that the AIC-type selector is asymptotically loss efficient, while the BIC-type selector is not. Our simulation results confirm these theoretical findings, and an empirical example is presented. Some technical proofs are given in the online supplementary material.
ISSN:0162-1459
1537-274X
DOI:10.1198/jasa.2009.tm08013