Questionable research practices among researchers in the most research‐productive management programs

• Published in: Journal of organizational behavior 2022-09, Vol.43 (7), p.1190-1208
• Main Authors: Kepes, Sven, Keener, Sheila K., McDaniel, Michael A., Hartman, Nathan S.
• Format: Article
• Language: English
• Subjects: Chrysalis Effect; Dissertations & theses; HARKing; Hypotheses; Organizational behavior; Organizational change; questionable research practices; research integrity; Statistical significance
• ISSN: 0894-3796; 1099-1379
• DOI: 10.1002/job.2623

Summary Questionable research practices (QRPs) among researchers have been a source of concern in many fields of study. QRPs are often used to enhance the probability of achieving statistical significance which affects the likelihood of a paper being published. Using a sample of researchers from 10 top research‐productive management programs, we compared hypotheses tested in dissertations to those tested in journal articles derived from those dissertations to draw inferences concerning the extent of engagement in QRPs. Results indicated that QRPs related to changes in sample size and covariates were associated with unsupported dissertation hypotheses becoming supported in journal articles. Researchers also tended to exclude unsupported dissertation hypotheses from journal articles. Likewise, results suggested that many article hypotheses may have been created after the results were known (i.e., HARKed). Articles from prestigious journals contained a higher percentage of potentially HARKed hypotheses than those from less well‐regarded journals. Finally, articles published in prestigious journals were associated with more QRP usage than less prestigious journals. QRPs increase in the percentage of supported hypotheses and result in effect sizes that likely overestimate population parameters. As such, results reported in articles published in our most prestigious journals may be less credible than previously believed.