Can Researchers’ Personal Characteristics Shape Their Statistical Inferences?

Researchers’ subjective judgments may affect the statistical results they obtain. This possibility is particularly stark in Bayesian hypothesis testing: To use this increasingly popular approach, researchers specify the effect size they are expecting (the “prior mean”), which is then incorporated in...

Full description

Saved in:
Bibliographic Details
Published in:Personality & social psychology bulletin 2021-06, Vol.47 (6), p.969-984
Main Authors: Dunn, Elizabeth W., Chen, Lihan, Proulx, Jason D. E., Ehrlinger, Joyce, Savalei, Victoria
Format: Article
Language:English
Subjects:
Bayesian analysis
Confidence
Gender differences
Hypotheses
Hypothesis testing
Personal characteristics
Psychology
Researcher subject relations
Researchers
Statistical inference
Statistics
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Researchers’ subjective judgments may affect the statistical results they obtain. This possibility is particularly stark in Bayesian hypothesis testing: To use this increasingly popular approach, researchers specify the effect size they are expecting (the “prior mean”), which is then incorporated into the final statistical results. Because the prior mean represents an expression of confidence that one is studying a large effect, we reasoned that scientists who are more confident in their research skills may be inclined to select larger prior means. Across two preregistered studies with more than 900 active researchers in psychology, we showed that more self-confident researchers selected larger prior means. We also found suggestive but somewhat inconsistent evidence that men may choose larger prior means than women, due in part to gender differences in researcher self-confidence. Our findings provide the first evidence that researchers’ personal characteristics might shape the statistical results they obtain with Bayesian hypothesis testing.
ISSN:0146-1672
1552-7433
DOI:10.1177/0146167220950522