ERIC Number: EJ1202757
Record Type: Journal
Publication Date: 2018-Dec
Pages: 27
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-2330-8516
EISSN: N/A
Agreement of Teachers on Evaluating Assessments of Learning Progressions in English Language Arts and Mathematics. Research Report. ETS RR-18-11
van Rijn, Peter; Graf, Edith Aurora; Arieli-Attali, Meirav; Song, Yi
ETS Research Report Series, Dec 2018
In this study, we explored the extent to which teachers agree on the ordering and separation of levels of two different learning progressions (LPs) in English language arts (ELA) and mathematics. In a panel meeting akin to a standard-setting procedure, we asked teachers to link the items and responses of summative educational assessments to LP levels. We evaluated four types of agreement among teachers, with test developers, and with trained raters. Although the results were quite mixed, agreement on the LP levels of mathematics items was generally better than that for ELA tasks. The agreement on the LP levels of ELA sample responses was generally better than that for mathematics. Implications of the results are discussed.
Descriptors: Teacher Attitudes, Student Evaluation, Summative Evaluation, Language Arts, Mathematics, Interrater Reliability, Test Construction, Learning Processes, Test Items, Difficulty Level, Mathematics Teachers, Secondary School Teachers, English Teachers
Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Publication Type: Journal Articles; Reports - Research
Education Level: Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: New Jersey; Pennsylvania
Grant or Contract Numbers: N/A