BDF: A new decision forest algorithm

• Published in: Information sciences 2021-08, Vol.569, p.687-705
• Main Authors: Adnan, Md Nasim, Ip, Ryan H.L., Bewong, Michael, Islam, Md Zahidul
• Format: Article
• Language: English
• Subjects: Classification; Decision forest; Decision tree; Random forest
• ISSN: 0020-0255; 1872-6291
• DOI: 10.1016/j.ins.2021.05.017

The foremost requirement for a decision forest to achieve better ensemble accuracy is building a set of accurate and diverse individual decision trees as base classifiers. Existing decision forest algorithms mainly differ from each other on how they induce diversity among the decision trees. At the same time, most of the drawbacks of existing algorithms originate from their induction processes of diversity. In this paper, we propose a new decision forest algorithm that is more balanced through effective synchronization between different sources of diversity. The proposed algorithm is balanced theoretically and empirically. We carried out experiments on 25 well-known data sets that are publicly available from the UCI Machine Learning Repository, to perform an extensive empirical evaluation. The experimental results indicate that the proposed algorithm has the best average ensemble accuracy rank of 1.8 compared to its closest competitor at 3.5. Using the Friedman and Bonferroni-Dunn tests, we also show that such an improvement is indeed statistically significant. In addition, the proposed algorithm is found to be competitive in terms of complexity and other relevant parameters.