A large‐scale assessment of temporal trends in meta‐analyses using systematic review reports from the Cochrane Library

• Published in: Research synthesis methods 2017-12, Vol.8 (4), p.404-415
• Main Authors: Fanshawe, Thomas R., Shaw, Luke F., Spence, Graeme T.
• Format: Article
• Language: English
• Subjects: Algorithms; Bayes Theorem; cumulative meta‐analysis; CUSUM charts; Extreme values; generalised least squares; Humans; Intervention; Least squares method; Least Squares Statistics; Least-Squares Analysis; Medical Research; Meta Analysis; Meta-Analysis as Topic; Models, Statistical; Odds Ratio; Outcomes of Treatment; Predictor Variables; Proportional Hazards Models; Proteus phenomenon; Quality Control; quality control charts; Reproducibility of Results; Research Design - trends; Research Reports; Risk; Studies; Systematic review; Systematic Reviews as Topic; temporal trend; Trend Analysis; Trends
• ISSN: 1759-2879; 1759-2887
• DOI: 10.1002/jrsm.1238

Introduction Previous studies suggest that many systematic reviews contain meta‐analyses that display temporal trends, such as the first study's result being more extreme than later studies' or a drift in the pooled estimate. We assessed the extent and characteristics of temporal trends using all Cochrane intervention reports published 2008‐2012. Methods We selected the largest meta‐analysis within each report and analysed trends using methods including a Z‐test (first versus subsequent estimates); generalised least squares; and cumulative sum charts. Predictors considered include meta‐analysis size and review group. Results Of 1288 meta‐analyses containing at least 4 studies, the point estimate from the first study was more extreme and in the same direction as the pooled estimate in 738 (57%), with a statistically significant difference (first versus subsequent) in 165 (13%). Generalised least squares indicated trends in 717 (56%); 18% of fixed effects analyses had at least one violation of cumulative sum limits. For some methods, meta‐analysis size was associated with temporal patterns and use of a random effects model, but there was no consistent association with review group. Conclusions All results suggest that more meta‐analyses demonstrate temporal patterns than would be expected by chance. Hence, assuming the standard meta‐analysis model without temporal trend is sometimes inappropriate. Factors associated with trends are likely to be context specific.