Runtime Optimizations for Tree-Based Machine Learning Models

Tree-based models have proven to be an effective solution for web ranking as well as other machine learning problems in diverse domains. This paper focuses on optimizing the runtime performance of applying such models to make predictions, specifically using gradient-boosted regression trees for lear...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on knowledge and data engineering 2014-09, Vol.26 (9), p.2281-2292
Main Authors: Asadi, Nima, Lin, Jimmy, de Vries, Arjen P.
Format: Article
Language:English
Subjects:
Arrays
Indexes
Information Storage and Retrieval
Information Technology and Systems
Learning to Rank
Optimization
Predictive models
Program processors
Regression tree analysis
Scalability and Efficiency
Web Search
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Tree-based models have proven to be an effective solution for web ranking as well as other machine learning problems in diverse domains. This paper focuses on optimizing the runtime performance of applying such models to make predictions, specifically using gradient-boosted regression trees for learning to rank. Although exceedingly simple conceptually, most implementations of tree-based models do not efficiently utilize modern superscalar processors. By laying out data structures in memory in a more cache-conscious fashion, removing branches from the execution flow using a technique called predication, and micro-batching predictions using a technique called vectorization, we are able to better exploit modern processor architectures. Experiments on synthetic data and on three standard learning-to-rank datasets show that our approach is significantly faster than standard implementations.
ISSN:1041-4347
1558-2191
DOI:10.1109/TKDE.2013.73