Misunderstandings about Q and 'Cochran's Q test' in meta-analysis

Many meta‐analyses report using ‘Cochran's Q test' to assess heterogeneity of effect‐size estimates from the individual studies. Some authors cite work by W. G. Cochran, without realizing that Cochran deliberately did not use Q itself to test for heterogeneity. Further, when heterogeneity...

Full description

Saved in:
Bibliographic Details
Published in:Statistics in medicine 2016-02, Vol.35 (4), p.485-495
Main Author: Hoaglin, David C.
Format: Article
Language:English
Subjects:
Approximation
Computer Simulation
Confidence intervals
DerSimonian-Laird procedure
Effect Modifier, Epidemiologic
heterogeneity
Humans
Medical statistics
Meta-analysis
Meta-Analysis as Topic
Models, Statistical
Numerical Analysis, Computer-Assisted
random-effects meta-analysis
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Many meta‐analyses report using ‘Cochran's Q test' to assess heterogeneity of effect‐size estimates from the individual studies. Some authors cite work by W. G. Cochran, without realizing that Cochran deliberately did not use Q itself to test for heterogeneity. Further, when heterogeneity is absent, the actual null distribution of Q is not the chi‐squared distribution assumed for ‘Cochran's Q test'. This paper reviews work by Cochran related to Q. It then discusses derivations of the asymptotic approximation for the null distribution of Q, as well as work that has derived finite‐sample moments and corresponding approximations for the cases of specific measures of effect size. Those results complicate implementation and interpretation of the popular heterogeneity index I2. Also, it turns out that the test‐based confidence intervals used with I2 are based on a fallacious approach. Software that outputs Q and I2 should use the appropriate reference value of Q for the particular measure of effect size and the current meta‐analysis. Q is a key element of the popular DerSimonian–Laird procedure for random‐effects meta‐analysis, but the assumptions of that procedure and related procedures do not reflect the actual behavior of Q and may introduce bias. The DerSimonian–Laird procedure should be regarded as unreliable. Copyright © 2015 John Wiley & Sons, Ltd.
ISSN:0277-6715
1097-0258
DOI:10.1002/sim.6632