Economic Analysis of Simulation Selection Problems

Ranking and selection procedures are standard methods for selecting the best of a finite number of simulated design alternatives based on a desired level of statistical evidence for correct selection. But the link between statistical significance and financial significance is indirect, and there has...

Full description

Saved in:
Bibliographic Details
Published in:Management science 2009-03, Vol.55 (3), p.421-437
Main Authors: Chick, Stephen E, Gans, Noah
Format: Article
Language:English
Subjects:
Applied sciences
Approximation
Bayesian analysis
Business metrics
Business studies
Computer simulation
Computer-generated environments
Decision theory. Utility theory
Determinism
Discounting
Economic analysis
Economic value
economics of simulation
Exact sciences and technology
Firm modelling
free boundary problem
Management science
Marginal costs
Mathematical statistics
Methods
Net present value
Operational research and scientific management
Operational research. Management science
Operations research
optimal stopping
Product design
Production management
ranking and selection
Selection
Simulation
Stochastic analysis
Stochastic models
Stopping distances
Studies
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Ranking and selection procedures are standard methods for selecting the best of a finite number of simulated design alternatives based on a desired level of statistical evidence for correct selection. But the link between statistical significance and financial significance is indirect, and there has been little or no research into it. This paper presents a new approach to the simulation selection problem, one that maximizes the expected net present value of decisions made when using stochastic simulation. We provide a framework for answering these managerial questions: When does a proposed system design, whose performance is unknown, merit the time and money needed to develop a simulation to infer its performance? For how long should the simulation analysis continue before a design is approved or rejected? We frame the simulation selection problem as a "stoppable" version of a Bayesian bandit problem that treats the ability to simulate as a real option prior to project implementation. For a single proposed system, we solve a free boundary problem for a heat equation that approximates the solution to a dynamic program that finds optimal simulation project stopping times and that answers the managerial questions. For multiple proposed systems, we extend previous Bayesian selection procedures to account for discounting and simulation-tool development costs.
ISSN:0025-1909
1526-5501
DOI:10.1287/mnsc.1080.0949