ERIC Number: ED562761
Record Type: Non-Journal
Publication Date: 2014
Pages: 19
Abstractor: ERIC
ISBN: N/A
ISSN: N/A
EISSN: N/A
Improving Balance in Regression Discontinuity Design by Matching: Estimating the Effect of Academic Probation after the First Year of College
Chi, Olivia L.; Dow, Aaron W.
Society for Research on Educational Effectiveness
This study focuses on how matching, a method of preprocessing data prior to estimation and analysis, can be used to reduce imbalance between treatment and control group in regression discontinuity design. To examine the effects of academic probation on student outcomes, researchers replicate and expand upon research conducted by Lindo, Sanders, and Oreopolous in their 2010 study "Ability, Gender, and Performance Standards: Evidence from Academic Probation." In replicating the results of Lindo et al. (2010), researchers find imbalance in observable pre-treatment characteristics between the treatment and control groups in the study. Such imbalance may indicate that randomization may not have been properly approximated through the regression discontinuity design. To improve balance and better approximate a randomized experiment, researchers preprocess Lindo et al.'s data set by performing exact matching on pre-treatment covariates, such as high school grade percentile and native language, and then re-estimate the impact of academic probation on student outcomes. After preprocessing the data through exact matching, the sample consists of a subset of the original sample (12,530), with 6,506 first year students in a large Canadian university made up of one central campus and two smaller satellite campuses. Results indicate that the effect sizes of academic probation on the decision to leave the university, as well as the effect on subsequent grades, were previously underestimated by Lindo et al. (2010). By utilizing the matching method of preprocessing data prior to estimation and analysis, imbalance was reduced between treatment and control groups in a regression discontinuity design. By improving balance, bias was reduced in the estimates of the effects of being placed on academic probation on student outcomes, and larger discouragement and encouragement effects were uncovered. Tables and figures are appended.
Descriptors: Regression (Statistics), Quasiexperimental Design, Nonparametric Statistics, Academic Probation, Undergraduate Students, Experimental Groups, Control Groups, Foreign Countries, Replication (Evaluation), Effect Size, Suspension, Incentives, Student Improvement, Grade Point Average
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail: inquiries@sree.org; Web site: http://www.sree.org
Publication Type: Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Location: Canada
Grant or Contract Numbers: N/A