An Optimal Constrained Pruning Strategy for Decision Trees

• Published in: INFORMS journal on computing 2009-12, Vol.21 (1), p.49-61
• Main Authors: Sherali, Hanif D, Hobeika, Antoine G, Jeenanunta, Chawalit
• Format: Article
• Language: English
• Subjects: Algorithms; analysis of algorithms; Classification; Data mining; Datasets; Decision analysis; Decision tree; Decision trees; Dependent variables; integer; Integer programming; Mathematical optimization; Methods; Optimization; programming; Properties; Studies; transportation; Variables
• ISSN: 1091-9856; 1526-5528; 1091-9856
• DOI: 10.1287/ijoc.1080.0278

This paper is concerned with the optimal constrained pruning of decision trees. We present a novel 0–1 programming model for pruning the tree to minimize some general penalty function based on the resulting leaf nodes, and show that this model possesses a totally unimodular structure that enables it to be solved as a shortest-path problem on an acyclic graph. Moreover, we prove that this problem can be solved in strongly polynomial time while incorporating an additional constraint on the number of residual leaf nodes. Furthermore, the framework of the proposed modeling approach renders it suitable to accommodate different (multiple) objective functions and side-constraints, and we identify various such modeling options that can be applied in practice. The developed methodology is illustrated using a numerical example to provide insights, and some computational results are presented to demonstrate the efficacy of solving generically constrained problems of this type. We also apply this technique to a large-scale transportation analysis and simulation system (TRANSIMS), and present related computational results using real data to exhibit the flexibility and effectiveness of the proposed approach.