ERIC Number: ED553415
Record Type: Non-Journal
Publication Date: 2014-Oct
Pages: 41
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Combining Multiple Performance Measures: Do Common Approaches Undermine Districts' Personnel Evaluation Systems? CALDER Working Paper No. 118
Hansen, Michael; Lemke, Mariann; Sorensen, Nicholas
National Center for Analysis of Longitudinal Data in Education Research (CALDER)
Teacher and principal evaluation systems now emerging in response to federal, state and/or local policy initiatives typically require that a component of teacher evaluation be based on multiple performance metrics, which must be combined to produce summative ratings of teacher effectiveness. Districts have utilized three common approaches to combine these multiple performance measures, all of which introduce bias and/or additional prediction error that was not present in the performance measures originally. This paper investigates whether the bias and error introduced by these approaches erodes the ability of evaluation systems to reliably identify high- and low-performing teachers. The analysis compares the expected differences in long-term teacher value-added among teachers identified as high- or low-performing under these three approaches, using simulated data based on estimated inter-correlations and reliability of measures in the Gates Foundation's Measures of Effective Teaching project. Based on the results of our simulation exercise presented here, we conclude these approaches can undermine the evaluation system's objectives in some contexts. Depending on the way these performance measures are actually combined to categorize teacher performance, the additional error and bias can be large enough to undermine the district's objectives. An appendix presents: (1) Correlation Matrix; and (2) Estimated Coefficients under the Error-Minimizing Approach.
Descriptors: Teacher Evaluation, Measures (Individuals), Error of Measurement, Teacher Effectiveness, Simulation, School Districts, Goodness of Fit, Evaluation Methods, Reliability, Principals
National Center for Analysis of Longitudinal Data in Education Research. American Institutes for Research, 1000 Thomas Jefferson Street NW, Washington, DC 20007. Tel: 202-403-5000; Fax: 202-403-5454; e-mail: inquiry@caldercenter.org; Web site: http://www.caldercenter.org
Publication Type: Reports - Research
Education Level: Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED)
Authoring Institution: National Center for Analysis of Longitudinal Data in Education Research (CALDER) at American Institutes for Research
Identifiers - Laws, Policies, & Programs: Elementary and Secondary Education Act; Race to the Top
IES Funded: Yes
Grant or Contract Numbers: R305C120008