Multivariate meta-analysis: the effect of ignoring within-study correlation

• Published in: Journal of the Royal Statistical Society. Series A, Statistics in society Statistics in society, 2009-10, Vol.172 (4), p.789-811
• Main Author: Riley, Richard D.
• Format: Article
• Language: English
• Subjects: Analytical estimating; Applications; Bivariate random-effects meta-analysis; Calculus of variations and optimal control; Correlation analysis; Correlations; Covariance; Data analysis; Datasets; Estimation bias; Exact sciences and technology; General topics; Mathematical analysis; Mathematics; Meta analysis; Missing data; Modeling; Multiple end points; Multiple outcomes; Multivariate analysis; Multivariate meta-analysis; Patients; Point estimators; Probability and statistics; Quantitative analysis; Sciences and techniques of general use; Sensitivity analysis; Simulation; Standard error; Statistical analysis; Statistical data; Statistical discrepancies; Statistical methods; Statistics; Unknown within-study correlation
• ISSN: 0964-1998; 1467-985X
• DOI: 10.1111/j.1467-985X.2008.00593.x

Multivariate meta-analysis allows the joint synthesis of summary estimates from multiple end points and accounts for their within-study and between-study correlation. Yet practitioners usually meta-analyse each end point independently. I examine the role of within-study correlation in multivariate meta-analysis, to elicit the consequences of ignoring it. Using analytic reasoning and a simulation study, the within-study correlation is shown to influence the 'borrowing of strength' across end points, and wrongly ignoring it gives meta-analysis results with generally inferior statistical properties; for example, on average it increases the mean-square error and standard error of pooled estimates, and for non-ignorable missing data it increases their bias. The influence of within-study correlation is only negligible when the within-study variation is small relative to the between-study variation, or when very small differences exist across studies in the within-study covariance matrices. The findings are demonstrated by applied examples within medicine, dentistry and education. Meta-analysts are thus encouraged to account for the correlation between end points. To facilitate this, I conclude by reviewing options for multivariate meta-analysis when within-study correlations are unknown; these include obtaining individual patient data, using external information, performing sensitivity analyses and using alternatively parameterized models.