ERIC Number: EJ1153666
Record Type: Journal
Publication Date: 2017
Pages: 36
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0125-2488
EISSN: N/A
Examining the Impact of Scoring Methods on the Institutional EFL Writing Assessment: A Turkish Perspective
Han, Turgay; Huang, Jinyan
PASAA: Journal of Language Teaching and Learning in Thailand, v53 p112-147 Jan-Jun 2017
Using generalizability (G-) theory and rater interviews as both quantitative and qualitative approaches, this study examined the impact of scoring methods (i.e., holistic versus analytic scoring) on the scoring variability and reliability of an EFL institutional writing assessment at a Turkish university. Ten raters were invited to rate 36 undergraduate argumentative essays first holistically and then analytically, with a three-week time interval. The quantitative results indicated that with proper rater training holistic scoring can produce as reliable and dependable assessment outcomes as analytic scoring. Similarly, the qualitative results revealed that all raters prefer using the holistic scoring method because it could help them not only assign fair and objective scores to essays but also facilitate their scoring process. Further, most raters agreed that the content of an essay was the most important factor that most affected their holistic scoring decision making of an essay. In contrast, all aspects of an essay (e.g., grammar, content, or organization) jointly affected their analytic scoring decision making of an essay. Important implications for EFL writing assessment professionals in the institutional assessment context are discussed.
Descriptors: English (Second Language), Second Language Learning, Second Language Instruction, Scoring, Holistic Approach, Evaluators, Essays, Generalization, Interviews, Qualitative Research, Writing Evaluation, Decision Making, Foreign Countries, Inferences, Statistical Analysis, Departments, English Literature, College Students, Rating Scales
Chulalongkorn University Language Institute. Prem Purachatra Building, Chulalongkom University, Phayathai Road, Pathumwan, Bangkok 10330, Thailand. Tel: +66-2-218-6092; Fax: +66-2-218-6104; e-mail: pasaa.editor@gmail.com; Web site: http://www.culi.chula.ac.th/publicationsonline/home_p1.php
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Turkey
Grant or Contract Numbers: N/A