NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1355666
Record Type: Journal
Publication Date: 2022-Dec
Pages: 35
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0002-8312
EISSN: EISSN-1935-1011
Examining Human and Automated Ratings of Elementary Students' Writing Quality: A Multivariate Generalizability Theory Application
Chen, Dandan; Hebert, Michael; Wilson, Joshua
American Educational Research Journal, v59 n6 p1122-1156 Dec 2022
We used multivariate generalizability theory to examine the reliability of hand-scoring and automated essay scoring (AES) and to identify how these scoring methods could be used in conjunction to optimize writing assessment. Students (n = 113) included subsamples of struggling writers and non-struggling writers in Grades 3-5 drawn from a larger study. Students wrote six essays across three genres. All essays were hand-scored by four raters and an AES system called Project Essay Grade (PEG). Both scoring methods were highly reliable, but PEG was more reliable for non-struggling students, while hand-scoring was more reliable for struggling students. We provide recommendations regarding ways of optimizing writing assessment and blending hand-scoring with AES.
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://bibliotheek.ehb.be:2993
Publication Type: Journal Articles; Reports - Research
Education Level: Elementary Education; Early Childhood Education; Grade 3; Primary Education; Grade 4; Intermediate Grades; Grade 5; Middle Schools
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A