NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
ERIC Number: ED604658
Record Type: Non-Journal
Publication Date: 2020-Jan-24
Pages: 31
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-
EISSN: N/A
Using Data from Randomized Trials to Assess the Likely Generalizability of Educational Treatment-Effect Estimates from Regression Discontinuity Designs
Bloom, Howard; Bell, Andrew; Reiman, Kayla
Grantee Submission
This article assesses the likely generalizability of educational treatment-effect estimates from regression discontinuity designs (RDDs) when treatment assignment is based on academic pretest scores. Our assessment uses data on outcome and pretest measures from six educational experiments, ranging from preschool through high school, to estimate RDD generalization bias. We then compare those estimates (reported as standardized effect sizes) with the What Works Clearinghouse (WWC) standard for acceptable bias size ([less than or equal] to 0.05[delta]) for two target populations, one spanning a half-standard deviation pretest-score range and another spanning a full-standard deviation pretest-score range. Our results meet this standard for all 18 study/outcome/pretest scenarios examined given the narrower target population, and for 15 scenarios given the broader target population. Fortunately, two of the three exceptions represent pronounced "ceiling effects" that can be identified empirically, making it possible to avoid unwarranted RDD generalizations, and the third exception is very close to the WWC standard. [This is the online version of an article published in "Journal of Research on Educational Effectiveness" (ISSN 1934-5747).]
Publication Type: Reports - Research
Education Level: Elementary Education; High Schools; Secondary Education
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED)
Authoring Institution: N/A
Identifiers - Location: North Carolina; New York (New York)
IES Funded: Yes
Grant or Contract Numbers: R305D140012