Short Comparative Interrupted Time Series Using Aggregate School-Level Data in Education Research

Short comparative interrupted times series (CITS) designs are increasingly being used in education research to assess the effectiveness of school-level interventions. These designs can be implemented relatively inexpensively, often drawing on publicly available data on aggregate school performance....

Full description

Saved in:
Bibliographic Details
Published in:Educational researcher 2018-06, Vol.47 (5), p.295-306
Main Authors: Hallberg, Kelly, Williams, Ryan, Swanlund, Andrew, Eno, Jared
Format: Article
Language:English
Subjects:
Comparative Analysis
Data Analysis
Education
Educational Change
Educational Improvement
Educational Research
Grants
Program Evaluation
Quasi-experimental methods
Quasiexperimental Design
Research Methodology
Researchers
School Effectiveness
Teacher Effectiveness
Time
Time series
Validity
Verbal aggression
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Short comparative interrupted times series (CITS) designs are increasingly being used in education research to assess the effectiveness of school-level interventions. These designs can be implemented relatively inexpensively, often drawing on publicly available data on aggregate school performance. However, the validity of this approach hinges on a variety of assumptions and design decisions that are not clearly outlined in the literature. This article aims to serve as a practice guide for applied researchers when deciding how and whether to use this approach. We begin by providing an overview of the assumptions needed to estimate causal effects using school-level data, common threats to validity faced in practice and what effects can and cannot be estimated using school-level data. We then examine two analytic decisions researchers face in practice when implementing the design: correctly modeling the pretreatment functional form, which is modeling the preintervention trend, and selecting comparison cases. We then illustrate the use of this design in practice drawing on data from the implementation of the school improvement grant (SIG) program in Ohio. We conclude with advice for applied researchers implementing this design.
ISSN:0013-189X
1935-102X
DOI:10.3102/0013189X18769302