MP-Boost: Minipatch Boosting via Adaptive Feature and Observation Sampling

Boosting methods are among the best generalpurpose and off-the-shelf machine learning approaches, gaining widespread popularity. In this paper, we seek to develop a boosting method that yields comparable accuracy to popular AdaBoost and gradient boosting methods, yet is faster computationally and wh...

Full description

Saved in:
Bibliographic Details
Main Authors: Toghani, Mohammad Taha, Allen, Genevera I.
Format: Conference Proceeding
Language:English
Subjects:
AdaBoost
Adaptive Sampling
Big Data
Boosting
Classification algorithms
Conferences
Internal Validation
Machine learning algorithms
Minipatch Learning
Probability distribution
Task analysis
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Boosting methods are among the best generalpurpose and off-the-shelf machine learning approaches, gaining widespread popularity. In this paper, we seek to develop a boosting method that yields comparable accuracy to popular AdaBoost and gradient boosting methods, yet is faster computationally and whose solution is more interpretable. We achieve this by developing MP-Boost, an algorithm loosely based on AdaBoost that learns by adaptively selecting small subsets of instances and features, or what we term minipatches (MP), at each iteration. By sequentially learning on tiny subsets of the data, our approach is computationally faster than classic boosting algorithms. MP-Boost upweights important features and challenging instances, hence adaptively selects the most relevant minipatches for learning. The learned probability distributions aid in interpretation of our method. We empirically demonstrate the interpretability and comparative accuracy of our algorithm on a variety of binary classification tasks.
ISSN:2375-933X
2375-9356
DOI:10.1109/BigComp51126.2021.00023