Bayesian reanalysis of null results reported in medicine: Strong yet variable evidence for the absence of treatment effects

Efficient medical progress requires that we know when a treatment effect is absent. We considered all 207 Original Articles published in the 2015 volume of the New England Journal of Medicine and found that 45 (21.7%) reported a null result for at least one of the primary outcome measures. Unfortuna...

Full description

Saved in:
Bibliographic Details
Published in:PloS one 2018-04, Vol.13 (4), p.e0195474-e0195474
Main Authors: Hoekstra, Rink, Monden, Rei, van Ravenzwaaij, Don, Wagenmakers, Eric-Jan
Format: Article
Language:English
Subjects:
Bayes Theorem
Bayesian analysis
Biology and Life Sciences
Chi-square test
Clinical trials
Confidence intervals
Contingency
Contingency tables
Data Interpretation, Statistical
Design of experiments
Factor Analysis, Statistical
Humans
Medical research
Medicine
Medicine and Health Sciences
Methods
Mortality
Null hypothesis
Outcome and process assessment (Medical care)
Periodicals as Topic
Physical Sciences
Research and Analysis Methods
Statistical analysis
Statistical hypothesis testing
Treatment Failure
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Efficient medical progress requires that we know when a treatment effect is absent. We considered all 207 Original Articles published in the 2015 volume of the New England Journal of Medicine and found that 45 (21.7%) reported a null result for at least one of the primary outcome measures. Unfortunately, standard statistical analyses are unable to quantify the degree to which these null results actually support the null hypothesis. Such quantification is possible, however, by conducting a Bayesian hypothesis test. Here we reanalyzed a subset of 43 null results from 36 articles using a default Bayesian test for contingency tables. This Bayesian reanalysis revealed that, on average, the reported null results provided strong evidence for the absence of an effect. However, the degree of this evidence is variable and cannot be reliably predicted from the p-value. For null results, sample size is a better (albeit imperfect) predictor for the strength of evidence in favor of the null hypothesis. Together, our findings suggest that (a) the reported null results generally correspond to strong evidence in favor of the null hypothesis; (b) a Bayesian hypothesis test can provide additional information to assist the interpretation of null results.
ISSN:1932-6203
1932-6203
DOI:10.1371/journal.pone.0195474