Investigation of Ensemble Variance as a Measure of True Forecast Variance

• Published in: Monthly weather review 2011-12, Vol.139 (12), p.3954-3963
• Main Authors: KOLCZYNSKI, Walter C, STAUFFER, David R, HAUPT, Sue Ellen, ALTMAN, Naomi S, AIJUN DENG
• Format: Article
• Language: English
• Subjects: Calibration; Earth, ocean, space; Exact sciences and technology; External geophysics; Independent sample; Meteorology; Stochastic models; Studies; Weather forecasting
• ISSN: 0027-0644; 1520-0493
• DOI: 10.1175/MWR-D-10-05081.1

The uncertainty in meteorological predictions is of interest for applications ranging from economic to recreational to public safety. One common method to estimate uncertainty is by using meteorological ensembles. These ensembles provide an easily quantifiable measure of the uncertainty in the forecast in the form of the ensemble variance. However, ensemble variance may not accurately reflect the actual uncertainty, so any measure of uncertainty derived from the ensemble should be calibrated to provide a more reliable estimate of the actual uncertainty in the forecast. A previous study introduced the linear variance calibration (LVC) as a simple method to determine the ensemble variance to error variance relationship and demonstrated this technique on real ensemble data. The LVC parameters, the slopes, and y intercepts, however, are generally different from the ideal values. This current study uses a stochastic model to examine the LVC in a controlled setting. The stochastic model is capable of simulating underdispersive and overdispersive ensembles as well as perfectly reliable ensembles. Because the underlying relationship is specified, LVC results can be compared to theoretical values of the slope and y intercept. Results indicate that all types of ensembles produce calibration slopes that are smaller than their theoretical values for ensemble sizes less than several hundred members, with corresponding y intercepts greater than their theoretical values. This indicates that all ensembles, even otherwise perfect ensembles, should be calibrated if the ensemble size is less than several hundred. In addition, it is shown that an adjustment factor can be computed for inadequate ensemble size. This adjustment factor is independent of the stochastic model and is applicable to any linear regression of error variance on ensemble variance. When applied to experiments using the stochastic model, the adjustment produces LVC parameters near their theoretical values for all ensemble sizes. Although the adjustment is unnecessary when applying LVC, it allows for a more accurate assessment of the reliability of ensembles, and a fair comparison of the reliability for differently sized ensembles.