Revisiting the utility of retrospective pre-post designs: The need for mixed-method pilot data

•Retrospective pretest designs should also include true pretest data collection.•Eliminating true pretests requires mixed-methods pilot evaluation.•We provide an applied example with corresponding interpretation. The retrospective pre-post design affords many benefits to program staff and, according...

Full description

Saved in:
Bibliographic Details
Published in:Evaluation and program planning 2018-10, Vol.70, p.83-89
Main Authors: Geldhof, G. John, Warner, Danielle A., Finders, Jennifer K., Thogmartin, Asia A., Clark, Adam, Longway, Kelly A.
Format: Article
Language:English
Subjects:
Child Rearing
Design
Education
Evaluation research
Evaluators
Instructional Effectiveness
Measurement
Mixed methods research
Parenthood education
Parenting education
Parenting skills
Parenting skills ladder
Parents & parenting
Questionnaires
Response bias
Response shift bias
Retrospective pretest
Skills
Thentest
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:•Retrospective pretest designs should also include true pretest data collection.•Eliminating true pretests requires mixed-methods pilot evaluation.•We provide an applied example with corresponding interpretation. The retrospective pre-post design affords many benefits to program staff and, accordingly, has piqued renewed interest among applied program evaluators. In particular, the field has witnessed increasing application of a post-program-only data collection strategy in which only posttest and retrospective pretest data are collected. A post-program-only assessment strategy takes considerably less time than is required for collecting pre-program data and presumably has the added benefit of eliminating the impact of response-shift bias. Response-shift bias occurs when the knowledge, skills, or experiences participants gain through program participation leads them to interpret questionnaire items in a qualitatively different manner at pretest versus posttest. In this article, we discuss the strengths and weaknesses associated with administering retrospective pretest assessments and underscore the importance of thoroughly evaluating any application of a retrospective measurement strategy prior to its broader implementation. We provide a practical illustration of this evaluation process using a mixed-method study that assesses one measure of parenting education program effectiveness—the Parenting Skills Ladder.
ISSN:0149-7189
1873-7870
DOI:10.1016/j.evalprogplan.2018.05.002