A Quantitative Method for Substantive Robustness Assessment

Empirical political science is not simply about reporting evidence; it is also about coming to conclusions on the basis of that evidence and acting on those conclusions. But whether a result is substantively significant––strong and certain enough to justify acting upon the belief that the null hypot...

Full description

Saved in:
Bibliographic Details
Published in:Political science research and methods 2015-01, Vol.3 (1), p.95-111
Main Authors: Esarey, Justin, Danneman, Nathan
Format: Article
Language:English
Subjects:
Advocacy
Cancer
Decision theory
Economic models
Economics
Errors
Hypotheses
Original Articles
Political Science
Politics
Quantitative analysis
Quantitative Methods
Researchers
Studies
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Empirical political science is not simply about reporting evidence; it is also about coming to conclusions on the basis of that evidence and acting on those conclusions. But whether a result is substantively significant––strong and certain enough to justify acting upon the belief that the null hypothesis is false––is difficult to objectively pin down, in part because different researchers have different standards for interpreting evidence. Instead, this article advocates judging results according to their “substantive robustness,” the degree to which a community with heterogeneous standards for interpreting evidence would agree that the result is substantively significant. This study illustrates how this can be done using Bayesian statistical decision techniques. Judging results in this way yields a tangible benefit: false positives are reduced without decreasing the power of the test, thus decreasing the error rate in published results.
ISSN:2049-8470
2049-8489
DOI:10.1017/psrm.2014.14