Good Research Practices for Comparative Effectiveness Research: Analytic Methods to Improve Causal Inference from Nonrandomized Studies of Treatment Effects Using Secondary Data Sources: The ISPOR Good Research Practices for Retrospective Database Analysis Task Force Report—Part III

• Published in: Value in health 2009-11, Vol.12 (8), p.1062-1073
• Main Authors: Johnson, Michael L., PhD, Crown, William, PhD, Martin, Bradley C., PharmD, PhD, Dormuth, Colin R., MA, ScD, Siebert, Uwe, MD, MPH, MSc, ScD
• Format: Article
• Language: English
• Subjects: causal inference; Causality; comparative effectiveness; Comparative Effectiveness Research - methods; Comparative Effectiveness Research - standards; Confounding Factors (Epidemiology); Data Interpretation, Statistical; Databases, Factual - statistics & numerical data; Decision Making; Doctors; Drug-Related Side Effects and Adverse Reactions; Economics, Pharmaceutical; Epidemiologic Methods; Health Policy; Humans; Internal Medicine; Internationality; Multivariate Analysis; nonrandomized studies; Outcome Assessment (Health Care) - methods; Policy making; Regression Analysis; research methods; Retrospective Studies; ROC Curve; secondary databases; Selection Bias; Sensitivity analysis; Stratification; Task forces
• ISSN: 1098-3015; 1524-4733
• DOI: 10.1111/j.1524-4733.2009.00602.x

ABSTRACT Objectives Most contemporary epidemiologic studies require complex analytical methods to adjust for bias and confounding. New methods are constantly being developed, and older more established methods are yet appropriate. Careful application of statistical analysis techniques can improve causal inference of comparative treatment effects from nonrandomized studies using secondary databases. A Task Force was formed to offer a review of the more recent developments in statistical control of confounding. Methods The Task Force was commissioned and a chair was selected by the ISPOR Board of Directors in October 2007. This Report, the third in this issue of the journal, addressed methods to improve causal inference of treatment effects for nonrandomized studies. Results The Task Force Report recommends general analytic techniques and specific best practices where consensus is reached including: use of stratification analysis before multivariable modeling, multivariable regression including model performance and diagnostic testing, propensity scoring, instrumental variable, and structural modeling techniques including marginal structural models, where appropriate for secondary data. Sensitivity analyses and discussion of extent of residual confounding are discussed. Conclusions Valid findings of causal therapeutic benefits can be produced from nonrandomized studies using an array of state-of-the-art analytic techniques. Improving the quality and uniformity of these studies will improve the value to patients, physicians, and policymakers worldwide.