Which significance test performs the best in climate simulations?

• Published in: Tellus. Series A, Dynamic meteorology and oceanography Dynamic meteorology and oceanography, 2014-01, Vol.66 (1), p.23139
• Main Authors: Decremer, Damien, Chung, Chul E., Ekman, Annica M. L., Brandefelt, Jenny
• Format: Article
• Language: English
• Subjects: Autocorrelation; Climate change; Climate models; climate noise; Computer simulation; Experiments; Full text; Gain; General circulation models; Global warming; internal variability; Marine; Oceanography; Precipitation; Robustness; significance test; Simulation; Statistical methods; Student's t-test; temporal correlation
• ISSN: 1600-0870; 0280-6495; 1600-0870
• DOI: 10.3402/tellusa.v66.23139

Climate change simulated with climate models needs a significance testing to establish the robustness of simulated climate change relative to model internal variability. Student's t-test has been the most popular significance testing technique despite more sophisticated techniques developed to address autocorrelation. We apply Student's t-test and four advanced techniques in establishing the significance of the average over 20 continuous-year simulations, and validate the performance of each technique using much longer (375-1000 yr) model simulations. We find that all the techniques tend to perform better in precipitation than in surface air temperature. A sizable performance gain using some of the advanced techniques is realised in the model Ts output portion with strong positive lag-1 yr autocorrelation (> + 0.6), but this gain disappears in precipitation. Furthermore, strong positive lag-1 yr autocorrelation is found to be very uncommon in climate model outputs. Thus, there is no reason to replace Student's t-test by the advanced techniques in most cases.