Analyzing the Influence of Dosage in Social Experiments With Application to the Supporting Healthy Marriage Program

In experimental evaluations of health and social programs, the role of dosage is rarely explored because researchers cannot usually randomize individuals to experience varying dosage levels. Instead, such evaluations reveal the average effects of exposure to an intervention, although program exposur...

Full description

Saved in:
Bibliographic Details
Published in:The American journal of evaluation 2018-06, Vol.39 (2), p.257-277
Main Authors: Moulton, Shawn R., Peck, Laura R., Greeney, Adam
Format: Article
Language:English
Subjects:
Comparative Analysis
Correlation
Evaluation Methods
Federal Programs
Financial Support
Intervention
Marital Satisfaction
Marital Status
Marriage
Prediction
Program Design
Program Effectiveness
Program Evaluation
Surveys
Workshops
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In experimental evaluations of health and social programs, the role of dosage is rarely explored because researchers cannot usually randomize individuals to experience varying dosage levels. Instead, such evaluations reveal the average effects of exposure to an intervention, although program exposure may vary widely. This article compares three analytic methods that are used to examine dosage effects: the Analysis of Symmetrically Predicted Endogenous Subgroups (ASPES) method developed by Peck, instrumental variables, and propensity score matching methods. The article provides an illustration using data from the Supporting Healthy Marriage (SHM) demonstration, which rigorously tested the effectiveness of a skills-based relationship education program that had high take-up (91%) but substantial variation in exposure thereafter. To understand the relationship between program dosage level and impact magnitude, we use the ASPES method to create subgroups defined by the number of workshop hours couples attended and estimate impacts for those groups. We find that the SHM program impact on study members who received any program services is similar in magnitude to the impact on those study members who received a high or low-to-moderate dose of program services. For experimental evaluations of federal programs, the ASPES method shows promise as a way to investigate the relationship between treatment dosage and impact magnitude, a relationship crucial to designers and funders of public programs seeking to maximize program effectiveness.
ISSN:1098-2140
1557-0878
DOI:10.1177/1098214017698566