Bayesian optimization with informative parametric models via sequential Monte Carlo

• Published in: Data-Centric Engineering (Online) 2022-01, Vol.3, Article e5
• Main Authors: Oliveira, Rafael, Scalzo, Richard, Kohn, Robert, Cripps, Sally, Hardman, Kyle, Close, John, Taghavi, Nasrin, Lemckert, Charles
• Format: Article
• Language: English
• Subjects: Algorithms; Approximation; Bayesian analysis; Bayesian inference; Bayesian optimization; Contaminants; Decision making; Disease control; Gaussian process; inverse problems; Neural networks; Optimization; Parametric statistics; Probabilistic models; sequential Monte Carlo; Statistical inference
• ISSN: 2632-6736; 2632-6736
• DOI: 10.1017/dce.2022.5

Bayesian optimization (BO) has been a successful approach to optimize expensive functions whose prior knowledge can be specified by means of a probabilistic model. Due to their expressiveness and tractable closed-form predictive distributions, Gaussian process (GP) surrogate models have been the default go-to choice when deriving BO frameworks. However, as nonparametric models, GPs offer very little in terms of interpretability and informative power when applied to model complex physical phenomena in scientific applications. In addition, the Gaussian assumption also limits the applicability of GPs to problems where the variables of interest may highly deviate from Gaussianity. In this article, we investigate an alternative modeling framework for BO which makes use of sequential Monte Carlo (SMC) to perform Bayesian inference with parametric models. We propose a BO algorithm to take advantage of SMC’s flexible posterior representations and provide methods to compensate for bias in the approximations and reduce particle degeneracy. Experimental results on simulated engineering applications in detecting water leaks and contaminant source localization are presented showing performance improvements over GP-based BO approaches.