Classification tree analysis using TARGET

Tree models are valuable tools for predictive modeling and data mining. Traditional tree-growing methodologies such as CART are known to suffer from problems including greediness, instability, and bias in split rule selection. Alternative tree methods, including Bayesian CART (Chipman et al., 1998;...

Full description

Saved in:
Bibliographic Details
Published in:Computational statistics & data analysis 2008, Vol.52 (3), p.1362-1372
Main Authors: Gray, J. Brian, Fan, Guangzhe
Format: Article
Language:English
Subjects:
CART
Combinatorics
Combinatorics. Ordered structures
Data mining
Decision tree
Exact sciences and technology
General topics
Genetic algorithm
Graph theory
Mathematics
Multivariate analysis
Numerical analysis
Numerical analysis. Scientific computation
Numerical methods in probability and statistics
Predictive modeling
Probability and statistics
Random forests
Sciences and techniques of general use
Statistics
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Tree models are valuable tools for predictive modeling and data mining. Traditional tree-growing methodologies such as CART are known to suffer from problems including greediness, instability, and bias in split rule selection. Alternative tree methods, including Bayesian CART (Chipman et al., 1998; Denison et al., 1998), random forests (Breiman, 2001a), bootstrap bumping (Tibshirani and Knight, 1999), QUEST (Loh and Shih, 1997), and CRUISE (Kim and Loh, 2001), have been proposed to resolve these issues from various aspects, but each has its own drawbacks. Gray and Fan (2003) described a genetic algorithm approach to constructing decision trees called tree analysis with randomly generated and evolved trees (TARGET) that performs a better search of the tree model space and largely resolves the problems with current tree modeling techniques. Utilizing the Bayesian information criterion ( BIC), Fan and Gray (2005) developed a version of TARGET for regression tree analysis. In this article, we consider the construction of classification trees using TARGET. We modify the BIC to handle a categorical response variable, but we also adjust its penalty component to better account for the model complexity of TARGET. We also incorporate the option of splitting rules based on linear combinations of two or three variables in TARGET, which greatly improves the prediction accuracy of TARGET trees. Comparisons of TARGET to existing methods, using simulated and real data sets, indicate that TARGET has advantages over these other approaches.
ISSN:0167-9473
1872-7352
DOI:10.1016/j.csda.2007.03.014