A Multitier Stacked Ensemble Algorithm for Improving Classification Accuracy

• Published in: Computing in science & engineering 2020-07, Vol.22 (4), p.74-85
• Main Authors: Pari, Ramalingam, Sandhya, Maheshwari, Sankar, Sharmila
• Format: Article
• Language: English
• Subjects: Algorithms; Classification algorithms; Classifiers; Computational modeling; Datasets; Machine learning; Prediction algorithms; Predictive models; Random variables; Support vector machines; Training data
• ISSN: 1521-9615; 1558-366X
• DOI: 10.1109/MCSE.2018.2873940

For real-world problems, ensemble learning performs better than the individual classifiers. This is true for datasets that have many instances closer to the decision boundary. Using a meta-learner to learn from the predictions of the base classifiers generalizes better. Hence, stacked ensemble (SE) is preferred over other ensemble methods. We extend the SE and propose a multitier stacked ensemble (MTSE) algorithm with three tiers, namely, a base tier, an ensemble tier, and a generalization tier. The base tier uses the traditional classifiers to predict the labels. Tenfold cross-validation is used to validate the models in the base tiers. The cross-validated predictions are combined using combination schemes in the next tier. The predictions from the ensemble tier are generalized using meta-learning to give the final prediction. When tested with 36 datasets, the MTSE gives superior performance over the SE. It achieves high accuracy and does not suffer from over-fitting/under-fitting.