Using Beta Coefficients to Impute Missing Correlations in Meta-Analysis Research: Reasons for Caution

Meta-analysis has become a well-accepted method for synthesizing empirical research about a given phenomenon. Many meta-analyses focus on synthesizing correlations across primary studies, but some primary studies do not report correlations. Peterson and Brown (2005) suggested that researchers could...

Full description

Saved in:
Bibliographic Details
Published in:Journal of applied psychology 2018-06, Vol.103 (6), p.644-658
Main Authors: Roth, Philip L., Le, Huy, Oh, In-Sue, Van Iddekinge, Chad H., Bobko, Philip
Format: Article
Language:English
Subjects:
Analysis
Bias
Correlation analysis
Deviation
Estimating techniques
Human
Meta Analysis
Simulation
Statistical Correlation
Statistical Data
Statistical Estimation
Studies
Systematic review
Citations: Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Meta-analysis has become a well-accepted method for synthesizing empirical research about a given phenomenon. Many meta-analyses focus on synthesizing correlations across primary studies, but some primary studies do not report correlations. Peterson and Brown (2005) suggested that researchers could use standardized regression weights (i.e., beta coefficients) to impute missing correlations. Indeed, their beta estimation procedures (BEPs) have been used in meta-analyses in a wide variety of fields. In this study, the authors evaluated the accuracy of BEPs in meta-analysis. We first examined how use of BEPs might affect results from a published meta-analysis. We then developed a series of Monte Carlo simulations that systematically compared the use of existing correlations (that were not missing) to data sets that incorporated BEPs (that impute missing correlations from corresponding beta coefficients). These simulations estimated ρ̄ (mean population correlation) and SDρ (true standard deviation) across a variety of meta-analytic conditions. Results from both the existing meta-analysis and the Monte Carlo simulations revealed that BEPs were associated with potentially large biases when estimating ρ̄ and even larger biases when estimating SDρ. Using only existing correlations often substantially outperformed use of BEPs and virtually never performed worse than BEPs. Overall, the authors urge a return to the standard practice of using only existing correlations in meta-analysis.
ISSN:0021-9010
1939-1854
DOI:10.1037/apl0000293