P-curving the Evidence: P-values Published in Human Factors (2017–2023)

• Published in: Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2024-09, Vol.68 (1), p.1700-1704
• Main Authors: Moussaoui, Jannah R., Hyk, Alina, Kuhn, Kali, Johnson, Jeremy, Fischer, Sophia, Hoxha, Briana, Munoz Gomez Andrade, Fernando, Smart-Zimmerman, Macy, Do, Andrew, Gutierrez, Boone, Gallik, Caitlin, McCarley, Jason S.
• Format: Article
• Language: English
• ISSN: 1071-1813; 2169-5067
• DOI: 10.1177/10711813241275505

Publication bias and questionable research practices can inflate the perceived credibility of reported scientific findings and lead to low replicability. This preregistered study aimed to estimate the evidentiary value of empirical findings published in the journal Human Factors (2017–2023) using two meta-analytic methods: p-curve analysis to examine the distribution of significant p-values and Bayesian mixture modeling of p-value distributions to gauge the degree of contamination from the null hypothesis. Empirical findings from 62 articles were included in the analyses. P-curve results indicated evidential value, ruling out high levels of selective reporting as an explanation for significant results. Mixture modeling estimated a 25% contamination rate by the null hypothesis among significant p-values. Results document the quality of empirical evidence reported in Human Factors.