The distribution of P -values in medical research articles suggested selective reporting associated with statistical significance

• Published in: Journal of clinical epidemiology 2017-07, Vol.87, p.70-77
• Main Authors: Perneger, Thomas V, Combescure, Christophe
• Format: Article
• Language: English
• Subjects: Bias; Biomedical Research - methods; Biomedical Research - statistics & numerical data; Confidence intervals; Cross-Sectional Studies; Epidemiology; Heterogeneity; Humans; Hypotheses; Internal Medicine; Irregularities; Medical research; Medical statistics; Null hypothesis; Observational studies; P-values; Periodicals as Topic - statistics & numerical data; Practice of research; Publication bias; Publication Bias - statistics & numerical data; Random variables; Science; Statistical analysis; Statistical significance; Statistical tests; Statistics as Topic - methods; Studies
• ISSN: 0895-4356; 1878-5921
• DOI: 10.1016/j.jclinepi.2017.04.003

Abstract Objectives Published P -values provide a window into the global enterprise of medical research. The aim of this study was to use the distribution of published P -values to estimate the relative frequencies of null and alternative hypotheses and to seek irregularities suggestive of publication bias. Study Design and Setting This cross-sectional study included P -values published in 120 medical research articles in 2016 (30 each from the BMJ , JAMA , Lancet , and New England Journal of Medicine ). The observed distribution of P -values was compared with expected distributions under the null hypothesis (i.e., uniform between 0 and 1) and the alternative hypothesis (strictly decreasing from 0 to 1). P -values were categorized according to conventional levels of statistical significance and in one-percent intervals. Results Among 4,158 recorded P -values, 26.1% were highly significant ( P