NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
ERIC Number: ED654813
Record Type: Non-Journal
Publication Date: 2024-May-22
Pages: 55
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Analysis of Regression Discontinuity Designs with a Binary Moderating Variable
Jason Schoeneberger; Christopher Rhoads
Grantee Submission
Regression discontinuity (RD) designs are increasingly used for causal evaluations. For example, if a student's need for a literacy intervention is determined by a low score on a past performance indicator and that intervention is provided to all students who fall below a cutoff on that indicator, an RD study can determine the intervention's main effects on subsequent outcomes with strong causal validity. However, the literature contains little guidance for conducting a moderation analysis within an RD context. Such moderation analyses are crucial in schools with very diverse and needy student populations because it may be rare to find a program that is equally effective with each of these subpopulations. In diverse schools, evaluators are interested in understanding not just whether a program generally works, but also in learning for whom it doesn't work. The current article focuses on moderation with a single binary variable. A simulation study compares: (1) different bandwidth selectors and (2) local polynomial regressions with interactions to local regressions on subsets of the data defined by values of the moderating variable. We find that existing bandwidth selectors optimized for main effects will choose bandwidths that are too small for moderation analysis. Additionally, choosing an optimal bandwidth for the subset regression approach may not be feasible for small to moderate sample sizes unless the moderator prevalence is near 0.5 and correlation with the assignment variable is small. We conclude that when sample sizes are small a global regression approach is likely to be preferred to utilizing bandwidth selectors optimized for main effects. The results from the simulation study are then followed by a illustrative practical example of applying these approaches in detecting important moderation effects in an evaluation study of the Accelerating Literacy For Adolescents Laboratory (ALFA LAB), a semester-long "extra help elective" class given to 9th graders entering high school with very low reading achievement. [This is the in-press version of an article published in "American Journal of Evaluation."]
Publication Type: Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED)
Authoring Institution: N/A
IES Funded: Yes
Grant or Contract Numbers: R305A180154