Resource Efficient Bayesian Optimization

We propose a resource-efficient Bayesian Optimization (BO) formulation that can provide the same convergence guarantees as traditional BO, while ensuring that the opti-mization makes efficient use of the available cloud or high-performance computing (HPC) resources. The paper is motivated by the fac...

Full description

Saved in:
Bibliographic Details
Main Authors: Juneja, Namit, Chandola, Varun, Zola, Jaroslaw, Wodo, Olga, Desai, Parth
Format: Conference Proceeding
Language:English
Subjects:
active learning
Bayes methods
Bayesian optimization
Cloud computing
Computational modeling
Costs
Expected Improvement
Gaussian processes
Machine learning
Optimization methods
Resource-efficient op-timization
Training
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We propose a resource-efficient Bayesian Optimization (BO) formulation that can provide the same convergence guarantees as traditional BO, while ensuring that the opti-mization makes efficient use of the available cloud or high-performance computing (HPC) resources. The paper is motivated by the fact that for many optimization problems that lend themselves well to BO, like hyper-parameter optimization for training large machine learning models, the single function evaluation cost depends on the model parameters as well as system parameters. The proposed Resource Efficient Bayesian Optimization (REBO) algorithm is a novel formulation that exploits this dependence and provides significant cost benefits for users who want to deploy BO on cloud and HPC resources that are characterized by availability of compute resources with varying costs and expected performance benefits. We demonstrate the effectiveness of REBO, in terms of convergence and resource-efficiency, on a variety of machine learning hyper-parameter optimization applications.
ISSN:2159-6190
DOI:10.1109/CLOUD62652.2024.00012