Impact Evaluation of Coronavirus Disease 2019 Policy: A Guide to Common Design Issues

Policy responses to coronavirus disease 2019 (COVID-19), particularly those related to nonpharmaceutical interventions, are unprecedented in scale and scope. However, evaluations of policy impacts require a complex combination of circumstance, study design, data, statistics, and analysis. Beyond the...

Full description

Saved in:
Bibliographic Details
Published in:American journal of epidemiology 2021-11, Vol.190 (11), p.2474-2486
Main Authors: Haber, Noah A, Clarke-Deelder, Emma, Salomon, Joshua A, Feller, Avi, Stuart, Elizabeth A
Format: Article
Language:English
Subjects:
Coronaviruses
COVID-19
Data analysis
Decision making
Epidemiology
Health policy
Infectious diseases
Statistical analysis
Viral diseases
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Policy responses to coronavirus disease 2019 (COVID-19), particularly those related to nonpharmaceutical interventions, are unprecedented in scale and scope. However, evaluations of policy impacts require a complex combination of circumstance, study design, data, statistics, and analysis. Beyond the issues that are faced for any policy, evaluation of COVID-19 policies is complicated by additional challenges related to infectious disease dynamics and a multiplicity of interventions. The methods needed for policy-level impact evaluation are not often used or taught in epidemiology, and they differ in important ways that may not be obvious. Methodological complications of policy evaluations can make it difficult for decision-makers and researchers to synthesize and evaluate the strength of the evidence in COVID-19 health policy papers. Here we 1) introduce the basic suite of policy-impact evaluation designs for observational data, including cross-sectional analyses, pre-/post- analyses, interrupted time-series analysis, and difference-in-differences analysis; 2) demonstrate key ways in which the requirements and assumptions underlying these designs are often violated in the context of COVID-19; and 3) provide decision-makers and reviewers with a conceptual and graphical guide to identifying these key violations. Our overall goal is to help epidemiologists, policy-makers, journal editors, journalists, researchers, and other research consumers understand and weigh the strengths and limitations of evidence.
ISSN:0002-9262
1476-6256
DOI:10.1093/aje/kwab185