A comparison of automated statistical quality control methods for error detection in historical radiosonde temperatures

• Published in: International journal of climatology 2016-01, Vol.36 (1), p.28-42
• Main Authors: Anderson, Ashley N., Browning, Joshua M., Comeaux, Joey, Hering, Amanda S., Nychka, Douglas
• Format: Article
• Language: English
• Subjects: Automation; Error correction & detection; quality control; radiosonde; skewness; temperature
• ISSN: 0899-8418; 1097-0088
• DOI: 10.1002/joc.4327

ABSTRACT This paper presents a comparison of statistical methods for automated random error detection in historic radiosonde temperatures through a rigorous simulation study. We simulate temperature data designed to mimic observed radiosonde temperature time series from ten climate regions and three pressure levels and contaminate this simulated data with errors that are similar to those occurring in the historical record. Robust estimates of centre and spread of the temperatures are used to standardize values and flag potentially erroneous observations, and five approaches for selecting subsets of observations upon which to base these estimates are tested. Two robust estimators, one of which is designed to work well for asymmetric distributions and gives different estimates of standard deviation for each tail of the distribution, are investigated. We use a logistic regression model to assess the effects of climate, pressure level, record length, contamination percentage, error type, and window size on each method combined with each estimator in terms of both correctly and incorrectly identified errors. Temperature distributions are not always symmetric, and based on the simulation, we find that the asymmetric estimator makes fewer mistakes in error identification, and we illustrate its application with a case study at a Russian station.