NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1270524
Record Type: Journal
Publication Date: 2020
Pages: 24
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1543-4303
EISSN: N/A
Available Date: N/A
Exploring the Impacts of Different Score Resolution Procedures on Person Fit and Estimated Achievement in Rater-Mediated Assessments
Wind, Stefanie A.; Walker, A. Adrienne
Language Assessment Quarterly, v17 n4 p362-385 2020
Scoring procedures for many rater-mediated performance assessments include score resolution procedures in which a third rater adjudicates discrepancies between two raters' ratings of the same performance. There are numerous approaches for calculating resolved scores that involve different combinations of the original and third ratings. Using data from two content-area writing assessments, we examined the impact of applying no resolution and applying three resolution techniques: TQ, Parity, and the Operational resolution method used in the assessment program on the values and psychometric quality of student achievement estimates. We observed substantial differences in person fit, a model-based measure of the psychometric quality of student achievement estimates, across methods. Our results highlight the importance of examining person fit as part of evaluating the psychometric quality of rater-mediated performance assessments in which score resolution procedures are employed.
Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A