ERIC Number: ED656771
Record Type: Non-Journal
Publication Date: 2021-Sep-28
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
A General Method for Adjusting Test Score Distributions to Account for Rescoring and Retesting
Sophie Litschwartz
Society for Research on Educational Effectiveness
Background/Context: Pass/fail standardized exams frequently selectively rescore failing exams and retest failing examinees. This practice distorts the test score distribution and can confuse those who do analysis on these distributions. In 2011, the Wall Street Journal showed large discontinuities in the New York City Regent test score distributions around the passing cutoff (Martinez & McGinty, 2011). The Wall Street Journal claimed the discontinuities were proof that teachers were manipulating or "scrubbing" test scores near the cutoff (Dee, Dobbie, Jacob, & Rockoff, 2019). However, it was official Regent policy to regrade all initially failing exams near the passing cutoff and this selective rescoring policy was sufficient to create a discontinuity by itself. Under Classical Test Theory, test scores are an additive combination of true proficiency and measurement error (Lord & Novick, 1968). Rescoring an exam redraws the grading measurement error. Only regrading failing exams creates a new distribution of scores for the failing examinees while leaving the distribution of the passing examinees untouched. The new error draw will cause some of the initially failing students to now pass, but no initially passing students will end up failing. When the new distribution for the initially failing students is combined with the old distribution of initially passing students, there is an excess of mass to the right of the cut point and missing mass to the left of the cut point. Therefore, a policy of selective rescoring failing students creates a discontinuity at the pass point on its own with no manipulation (Figure 1).
Descriptors: Standardized Tests, Pass Fail Grading, Scoring Rubrics, Scoring Formulas, Scoring, Cutting Scores, Test Interpretation, Test Reliability, Statistical Distributions, Test Theory
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Descriptive
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Location: New York (New York); New York
Grant or Contract Numbers: N/A