The Bayesian Lasso

• Published in: Journal of the American Statistical Association 2008-06, Vol.103 (482), p.681-686
• Main Authors: Park, Trevor, Casella, George
• Format: Article
• Language: English
• Subjects: Algorithms; Analytical estimating; Applications; Bayes estimators; Bayesian analysis; Bayesian method; Diabetes; Empirical Bayes; Estimation methods; Exact sciences and technology; General topics; Gibbs sampler; Hierarchical model; Hierarchies; Inverse Gaussian; Least squares; Linear inference, regression; Linear models; Linear regression; Logic and foundations; Mathematical logic, foundations, set theory; Mathematics; Maximum likelihood estimation; Musical intervals; Normal distribution; Parameter estimation; Parameter modification; Penalized regression; Probability and statistics; Recursion theory; Regression; Regression analysis; Robustness (mathematics); Scale mixture of normals; Sciences and techniques of general use; Statistical analysis; Statistical discrepancies; Statistical median; Statistical methods; Statistics; Structural hierarchy; Theory and Methods
• ISSN: 0162-1459; 1537-274X
• DOI: 10.1198/016214508000000337

The Lasso estimate for linear regression parameters can be interpreted as a Bayesian posterior mode estimate when the regression parameters have independent Laplace (i.e., double-exponential) priors. Gibbs sampling from this posterior is possible using an expanded hierarchy with conjugate normal priors for the regression parameters and independent exponential priors on their variances. A connection with the inverse-Gaussian distribution provides tractable full conditional distributions. The Bayesian Lasso provides interval estimates (Bayesian credible intervals) that can guide variable selection. Moreover, the structure of the hierarchical model provides both Bayesian and likelihood methods for selecting the Lasso parameter. Slight modifications lead to Bayesian versions of other Lasso-related estimation methods, including bridge regression and a robust variant.