Squeezing the Balloon: Propensity Scores and Unmeasured Covariate Balance

• Published in: Health services research 2013-08, Vol.48 (4), p.1487-1507
• Main Authors: Brooks, John M., Ohsfeldt, Robert L.
• Format: Article
• Language: English
• Subjects: Algorithms; Analysis; assumptions; binning; covariate balance; Health aspects; Health care delivery; Health Services Research - methods; Humans; Imbalance; matching; Medical care; Medical care quality; Medical research; Methods; Models, Theoretical; Observational research; Observational studies; Outcome and Process Assessment (Health Care) - methods; Outcome and Process Assessment (Health Care) - statistics & numerical data; Propensity Score; Propensity scores; Quality management; Quality of care; Randomized Controlled Trials as Topic - methods; Simulation; Studies; Treatment outcome; Treatment preferences; Untreated
• ISSN: 0017-9124; 1475-6773
• DOI: 10.1111/1475-6773.12020

Objective To assess the covariate balancing properties of propensity score‐based algorithms in which covariates affecting treatment choice are both measured and unmeasured. Data Sources/Study Setting A simulation model of treatment choice and outcome. Study Design Simulation. Data Collection/Extraction Methods Eight simulation scenarios varied with the values placed on measured and unmeasured covariates and the strength of the relationships between the measured and unmeasured covariates. The balance of both measured and unmeasured covariates was compared across patients either grouped or reweighted by propensity scores methods. Principal Findings Propensity score algorithms require unmeasured covariate variation that is unrelated to measured covariates, and they exacerbate the imbalance in this variation between treated and untreated patients relative to the full unweighted sample. Conclusions The balance of measured covariates between treated and untreated patients has opposite implications for unmeasured covariates in randomized and observational studies. Measured covariate balance between treated and untreated patients in randomized studies reinforces the notion that all covariates are balanced. In contrast, forced balance of measured covariates using propensity score methods in observational studies exacerbates the imbalance in the independent portion of the variation in the unmeasured covariates, which can be likened to squeezing a balloon. If the unmeasured covariates affecting treatment choice are confounders, propensity score methods can exacerbate the bias in treatment effect estimates.