NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: ED657174
Record Type: Non-Journal
Publication Date: 2021-Sep-29
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
The Effect of Teacher Evaluation on Academic Achievement and Attainment
Joshua Bleiberg; Eric J. Brunner; Erica Harbatkin; Matthew Kraft; Matthew G. Springer
Society for Research on Educational Effectiveness
Context: From 2009 through 2017, the vast majority of states implemented major reforms to their teacher evaluation systems (Bleiberg & Harbatkin, 2016). State adoption occurred in part due to Race to the Top (Howell & Magazinnik, 2017), which rewarded states for implementing teacher evaluation systems under the theory that generating better information about teacher performance could improve teacher quality (Hallgren, James-Burdumy, & Perez-Johnson, 2014). Purpose and Research Questions: Researchers examining teacher evaluation systems in specific district contexts have found mixed evidence on whether teacher evaluation influenced student achievement (Adnot et al, 2017; Dee & Wyckoff, 2015; Stecher et al., 2018), or effected teachers' self-reported professional improvement activities (Koedel et al, 2018). The generalizability of these results is limited to the contexts of the districts or states studied and the unique characteristics of those settings. The staggered timing in the uptake of these reforms across states provides a unique opportunity to measure the effect of these evaluation systems on student achievement and educational attainment. Additionally, little is understood about the relative merits of different teacher evaluation reforms. We ask the following research questions: What was the effect of teacher evaluation on academic outcomes? What was the effect of teacher evaluation on high school graduation and college enrollment? Setting, Population, and Intervention: We measure state level teacher evaluation reform using the measure available in Kraft et al (2020). We consider a state treated when the new evaluation system was implemented statewide for the first time. 46 states implement teacher evaluation throughout the country. Only California, Iowa, Montana, Nebraska, Vermont, and Wyoming do not implement teacher evaluation. Washington, D.C., implemented in 2009 and the remaining states implemented between 2012 to 2017. State reforms peaked in 2014, when 13 states implemented teacher evaluation. Research Design: We investigate the effect of teacher evaluation using the national student outcome data in the Stanford Education Data Archive (SEDA). Additionally, we examine whether teacher evaluation had any effect in states that implemented the reform with a high level of fidelity. The SEDA includes district level test scores for third through eighth grade, from 2009 to 2018, and for ELA and math. We begin by estimating Difference-in-Differences (DiD) models following: Y[subscript dgst]=[beta][subscript 1]Tch_Eval[subscript dst]+Year[subscript t]X([rho]X[subscript dt=2009]+[phi]W[subscript dt=2009])[alpha][subscript d]+[delta][subscript g]+[theta][subscript t]+[mu][subscript dsgt] where is an outcome for each school district d, state s, grade g, and year t equals one for states that implemented teacher evaluation and zero otherwise is a linear time trend that we multiply with each of the covariate vectors. X' is a vector of baseline (2009) district characteristics that includes: district race/ethnicity, urbanicity, student/teacher ratio, and per pupil instructional spending in thousands. W' is a vector of baseline (2009) socio-economic status proxies that includes: GDP, poverty rate, and the unemployment rate. Z' is a vector of baseline (2009) ELA score, baseline (2009) math score, district school improvement status (2007), mean district proficiency rate (2007). Every model includes district fixed effects ([alpha]), grade fixed effects ([delta]), and year fixed effects (theta). Standard errors are clustered by state. Analysis: To explore the non-parametric evolution of any treatment effects we specify an event study. The pre-treatment differences in academic and attainment outcomes are both statistically insignificant and substantively small. Additionally, there is no clear visual decline in the point estimates across the pre-treatment periods. To increase our power to detect effects we then estimate DiD models that pool across the post-treatment periods. We first estimate the effect of teacher evaluation on math and ELA test scores contained in SEDA. We subsequently estimate state-level DiD models using data from the American Community Survey on High School graduation and college enrollment. Results: Our results indicate teacher evaluation had no effect on student outcomes, on average, nationally. We can rule out positive effects as small as 0.01 SD in math and 0.005 SD in ELA. These findings are robust to a range of specifications including adding state specific linear trends and controlling for time-varying education reforms. The effect of teacher evaluation on earning High School diplomas and college enrollment is indistinguishable from zero. The event study models are precise enough to rule out a 3 percent increase in students with High School diplomas and about a 5 percent increase in college enrollment. We find precisely estimated null effects for states that implemented teacher evaluation rigorously and states that did not. We also show that teacher evaluation had a null effect schools that serve academically vulnerable students (e.g., Black, Hispanic, impoverished communities). Conclusion: The federal government and states invested hundreds of millions in teacher evaluation systems. Mandating districts to enact teacher evaluation likely contributed to weak implementation that accounts in part for the null effects we find. Teacher evaluation appears to have had a positive effect in specific school districts (Adnot et al., 2017; Dee & Wyckoff, 2015), but on average throughout the entire country the policy had no effect.
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: High Schools; Secondary Education; Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Grant or Contract Numbers: N/A