Ranking and Contextual Selection

• Published in: Operations research 2024-10
• Main Authors: Keslin, Gregory, Nelson, Barry L., Pagnoncelli, Bernardo, Plumlee, Matthew, Rahimian, Hamed
• Format: Article
• Language: English
• Subjects: experiment design; nonparametric; Simulation; statistical analysis
• ISSN: 0030-364X; 1526-5463
• DOI: 10.1287/opre.2023.0378

Stochastic simulation is a powerful tool for discovering system design decisions that are the best possible (optimal) when averaged over real-world uncertainty. However, in applications such as personalized medicine and web content optimization, even better decisions can be made if they are tailored to specific, contemporaneous covariate information, such as patient health history and user reading habits. Unfortunately, in these and similar applications, there is no time to perform a refined simulation optimization. In “Ranking and Contextual Selection,” Keslin, Nelson, Pagnoncelli, Plumlee, and Rahimian use off-the-shelf simulation optimization methods to create a database of covariates and associated decisions that form a covariate-to-decision classifier and an upper confidence bound on its optimality gap when applied to covariates not in the database. A realistic example of web page assortment optimization is presented using a data set from Yahoo!. This paper proposes a new ranking-and-selection procedure, called ranking and contextual selection, in which covariates provide context for data-driven decisions. Our procedure optimizes over a set of covariate design points off-line and then, given an actual observation of the covariate, makes an online decision based on classification—a distinctly new approach. We prove the existence of an experimental design that yields a pointwise probability of good selection guarantee and derive a postexperiment assessment of our procedure that provides an optimality gap upper bound with guaranteed coverage for decisions with respect to future covariates. We illustrate ranking and contextual selection with an application to assortment optimization using data available from Yahoo!. Funding: This work was supported by the National Science Foundation [Grant CMMI-2206973]. Supplemental Material: This article includes an online appendix and computer code and data supporting the study’s findings at https://doi.org/10.1287/opre.2023.0378 .