Two-point methods for assessing variability in simulation output

• Published in: Journal of statistical computation and simulation 1998-05, Vol.60 (3), p.183-205
• Main Authors: Cheng, Russell C. H., Holland, Wayne
• Format: Article
• Language: English
• Subjects: Exact sciences and technology; Experimental design; Mathematics; Nonparametric inference; Parameter estimation; Probability and statistics; Sciences and techniques of general use; sensitivity analysis; simulation of computer networks; Statistics; uncertainty analysis
• ISSN: 0094-9655; 1563-5163
• DOI: 10.1080/00949659808811887

In simulation experiments, the form of the distribution of input variables is often not known precisely. The simulation output then contains two sources of variation: that caused by uncertainty in estimating unknown parameters, and that caused by the inclusion of random variation within the simulation model itself. Cheng and Holland (1996) have shown how the classical method of statistical differential analysis (often called the δ-method) can be used to assess the degree of variability arising from each source. The disadvantage of the δ-method is that the computational effort needed for this increases linearly with the number of unknown parameters. In this paper it is shown that the method can be modified to assess the combined effect on the response output of variation in all the parameters by making most simulation replications at just twosettings of parameter values, making the method substantially independent of the number of unknown parameters. Thus, for problems where this number is large, such two-point methods are substantially more efficient than the unmodified δ-method. For illustration, simulation results on the operation of two different computer networks are given. The workload in assessing the accuracy of estimates using the proposed two-point methods is compared with that using the δ-method, showing the large efficiency gains possible using the two-point method.