ERIC Number: ED563464
Record Type: Non-Journal
Publication Date: 2011-Apr
Pages: 13
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Aligning Items and Achievement Levels: A Study Comparing Expert Judgments
Kaliski, Pamela; Huff, Kristen; Barry, Carol
College Board
For educational achievement tests that employ multiple-choice (MC) items and aim to reliably classify students into performance categories, it is critical to design MC items that are capable of discriminating student performance according to the stated achievement levels. This is accomplished, in part, by clearly understanding how item design features align with each of the achievement level descriptions. Difficulty drivers are defined as specific item features that are related to the difficulty of an item. The purpose of this research study was to pilot a methodology for working with subject matter experts to identify what item features drive item difficulty, and to determine specific features that drive item difficulty on a summative large scale assessment used in high school social science and history classrooms. Results of the study indicated that although this methodology was useful for gathering such information, modifications could be made in the future to better elicit the desired information about difficulty drivers. [Slides presented at the Annual Meeting of the American Educational Research Association (AERA) and National Council on Measurement in Education (NCME) in New Orleans, LA in April 2011.]
Descriptors: Alignment (Education), Academic Achievement, Expertise, Evaluative Thinking, Achievement Tests, Multiple Choice Tests, Test Construction, Instructional Design, Pilot Projects, Test Items, Item Analysis, Comparative Analysis, Difficulty Level, Summative Evaluation, Educational Assessment, High School Students, Student Evaluation, Social Sciences, History, Evaluation Methods, Data Collection, Advanced Placement, Taxonomy
College Board. 250 Vesey Street, New York, NY 10281. Tel: 212-713-8000; e-mail: research@collegeboard.org; Web site: http://research.collegeboard.org
Publication Type: Non-Print Media; Reference Materials - General
Education Level: High Schools; Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: College Board
Grant or Contract Numbers: N/A