NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ976523
Record Type: Journal
Publication Date: 2012
Pages: 23
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1543-4303
EISSN: N/A
Available Date: N/A
Operational Rater Types in Writing Assessment: Linking Rater Cognition to Rater Behavior
Eckes, Thomas
Language Assessment Quarterly, v9 n3 p270-292 2012
This research investigated the relation between rater cognition and rater behavior, focusing on differential severity/leniency regarding criteria. Participants comprised a sample of 18 raters examined in a previous rater cognition study (Eckes, 2008b). These raters, who were known to differ widely in their perceptions of criterion importance, provided ratings of live examinee writing performance. Based on these ratings, criterion-related bias measures were estimated using many-facet Rasch measurement. A cluster analysis of bias measures yielded four operational rater types. Each type was characterized by a distinct pattern of differentially severe or lenient ratings on particular criteria. The observed bias patterns were related to differential perceptions of criterion importance: Criteria perceived as highly important were more closely associated with severe ratings, and criteria perceived as less important were more closely associated with lenient ratings. Implications of the demonstrated link between rater cognition and rater behavior for future research into the nature of rater bias are discussed. (Contains 4 tables, 3 figures, and 3 footnotes.)
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A