NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: EJ1312965
Record Type: Journal
Publication Date: 2021
Pages: 15
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-1562-0506
EISSN: N/A
Applying Generalizability Theory in Language Testing: Comparing Nested and Crossed Scoring Designs in the Assessment of Speaking Skills
Polat, Murat; Turhan, Nihan Sölpük
International Journal of Curriculum and Instruction, v13 n3 p3344-3358 2021
Scoring language learners' speaking skills is open to a number of measurement errors since raters' personal judgements could involve in the process. Different grading designs in which raters score a student's whole speaking skills or a specific dimension of the speaking performance could be settled to control and minimize the amount of the error in grading foreign language speaking skills. Therefore, the present study aims to compare G and Phi coefficients gained from the scores of a full factorial (fully crossed) model versus a nested model where rubric components are nested in graders. Four experienced raters and 116 intermediate level language learners studying at a Turkish state university's language school voluntarily participated in this exploratory study. Findings revealed that the G and Phi values obtained with a full factorial grading model were higher. In addition, checking the variance components according to the source of variation, the variance associated with the student's main effect of the full factorial pattern was higher, while the variance value of the residual effect was lower. These findings reveal that full factorial designs could generate more reliable results in speaking exams, thus, it is recommended for language schools to implement the full factorial design in speaking exams when practical conditions such as enough time or sufficient number of graders are available.
World Council for Curriculum and Instruction. California School of Education, Alliant International University, 10455 Pomerado Road, San Diego, CA 92131. Web site: http://ijci.wcci-international.org/index.php/IJCI/about
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Turkey
Grant or Contract Numbers: N/A