Assessing variability in results in systematic reviews of diagnostic studies

• Published in: BMC medical research methodology 2016-01, Vol.16 (6), p.6-6, Article 6
• Main Authors: Naaktgeboren, Christiana A, Ochodo, Eleanor A, Van Enst, Wynanda A, de Groot, Joris A H, Hooft, Lotty, Leeflang, Mariska M G, Bossuyt, Patrick M, Moons, Karel G M, Reitsma, Johannes B
• Format: Article
• Language: English
• Subjects: Accuracy; Analysis; Analysis of Variance; Bias (epidemiology); Clinical trials; Confidence intervals; Data interpretation; Diagnostic techniques and procedures/standards; Diagnostic Tests, Routine - methods; Diagnostic Tests, Routine - standards; Diagnostic Tests, Routine - statistics & numerical data; Humans; Medical care; Meta-analysis; Meta-Analysis as Topic; Publications - standards; Publications - statistics & numerical data; Reproducibility of Results; Review Literature as Topic; ROC Curve; Sensitivity and specificity; Statistical; Studies; Taiwan
• ISSN: 1471-2288; 1471-2288
• DOI: 10.1186/s12874-016-0108-4

To describe approaches used in systematic reviews of diagnostic test accuracy studies for assessing variability in estimates of accuracy between studies and to provide guidance in this area. Meta-analyses of diagnostic test accuracy studies published between May and September 2012 were systematically identified. Information on how the variability in results was investigated was extracted. Of the 53 meta-analyses included in the review, most (n=48; 91%) presented variability in diagnostic accuracy estimates visually either through forest plots or ROC plots and the majority (n=40; 75%) presented a test or statistical measure for the variability. Twenty-eight reviews (53%) tested for variability beyond chance using Cochran's Q test and 31 (58%) reviews quantified it with I(2). 7 reviews (13%) presented between-study variance estimates (τ(2)) from random effects models and 3 of these presented a prediction interval or ellipse to facilitate interpretation. Half of all the meta-analyses specified what was considered a significant amount of variability (n=24; 49%). Approaches to assessing variability in estimates of accuracy varied widely between diagnostic test accuracy reviews and there is room for improvement. We provide initial guidance, complemented by an overview of the currently available approaches.