“Magnitude-based Inference”: A Statistical Review

• Published in: Medicine and science in sports and exercise 2015-04, Vol.47 (4), p.874-884
• Main Authors: Welsh, Alan H, Knight, Emma J
• Format: Article
• Language: English
• Subjects: Bayes Theorem; Data Interpretation, Statistical; Humans; SPECIAL COMMUNICATIONS: Invited; Sports Medicine - statistics & numerical data
• ISSN: 0195-9131; 1530-0315
• DOI: 10.1249/MSS.0000000000000451

PURPOSEWe consider “magnitude-based inference” and its interpretation by examining in detail its use in the problem of comparing two means. METHODSWe extract from the spreadsheets, which are provided to users of the analysis (http://www.sportsci.org/), a precise description of how “magnitude-based inference” is implemented. We compare the implemented version of the method with general descriptions of it and interpret the method in familiar statistical terms. RESULTS AND CONCLUSIONSWe show that “magnitude-based inference” is not a progressive improvement on modern statistics. The additional probabilities introduced are not directly related to the confidence interval but, rather, are interpretable either as P values for two different nonstandard tests (for different null hypotheses) or as approximate Bayesian calculations, which also lead to a type of test. We also discuss sample size calculations associated with “magnitude-based inference” and show that the substantial reduction in sample sizes claimed for the method (30% of the sample size obtained from standard frequentist calculations) is not justifiable so the sample size calculations should not be used. Rather than using “magnitude-based inference,” a better solution is to be realistic about the limitations of the data and use either confidence intervals or a fully Bayesian analysis.