Stochastic Tree Ensembles for Regularized Nonlinear Regression

This article develops a novel stochastic tree ensemble method for nonlinear regression, referred to as accelerated Bayesian additive regression trees, or XBART. By combining regularization and stochastic search strategies from Bayesian modeling with computationally efficient techniques from recursiv...

Full description

Saved in:
Bibliographic Details
Published in:Journal of the American Statistical Association 2023-01, Vol.118 (541), p.551-570
Main Authors: He, Jingyu, Hahn, P. Richard
Format: Article
Language:English
Subjects:
Algorithms
Bayesian
Bayesian analysis
Machine learning
Markov chain Monte Carlo
Markov chains
Neural networks
Regression analysis
Regression trees
Regularization
Search strategies
Simulation
Statistical methods
Statistics
Supervised learning
Trees
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This article develops a novel stochastic tree ensemble method for nonlinear regression, referred to as accelerated Bayesian additive regression trees, or XBART. By combining regularization and stochastic search strategies from Bayesian modeling with computationally efficient techniques from recursive partitioning algorithms, XBART attains state-of-the-art performance at prediction and function estimation. Simulation studies demonstrate that XBART provides accurate point-wise estimates of the mean function and does so faster than popular alternatives, such as BART, XGBoost, and neural networks (using Keras) on a variety of test functions. Additionally, it is demonstrated that using XBART to initialize the standard BART MCMC algorithm considerably improves credible interval coverage and reduces total run-time. Finally, two basic theoretical results are established: the single tree version of the model is asymptotically consistent and the Markov chain produced by the ensemble version of the algorithm has a unique stationary distribution.
ISSN:0162-1459
1537-274X
DOI:10.1080/01621459.2021.1942012