P-Curve: A Key to the File-Drawer

Because scientists tend to report only studies (publication bias) or analyses (p-hacking) that "work," readers must ask, "Are these effects true, or do they merely reflect selective reporting?" We introduce p-curve as a way to answer this question. P-curve is the distribution of...

Full description

Saved in:
Bibliographic Details
Published in:Journal of experimental psychology. General 2014-04, Vol.143 (2), p.534-547
Main Authors: Simonsohn, Uri, Nelson, Leif D., Simmons, Joseph P.
Format: Article
Language:English
Subjects:
Bias
Biological and medical sciences
Data Interpretation, Statistical
Experimental psychology
Fundamental and applied biological sciences. Psychology
Humans
Hypothesis Testing
Models, Statistical
Psychology
Psychology, Experimental
Psychology. Psychoanalysis. Psychiatry
Psychology. Psychophysiology
Psychometrics. Statistics. Methodology
Publication Bias
Reproducibility of Results
Scientific Communication
Statistics
Statistics as Topic
Statistics. Mathematics
Citations: Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Because scientists tend to report only studies (publication bias) or analyses (p-hacking) that "work," readers must ask, "Are these effects true, or do they merely reflect selective reporting?" We introduce p-curve as a way to answer this question. P-curve is the distribution of statistically significant p values for a set of studies (ps < .05). Because only true effects are expected to generate right-skewed p-curves-containing more low (.01s) than high (.04s) significant p values-only right-skewed p-curves are diagnostic of evidential value. By telling us whether we can rule out selective reporting as the sole explanation for a set of findings, p-curve offers a solution to the age-old inferential problems caused by file-drawers of failed studies and analyses.
ISSN:0096-3445
1939-2222
DOI:10.1037/a0033242