ERIC Number: EJ1314034
Record Type: Journal
Publication Date: 2021-Oct
Pages: 25
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0265-5322
EISSN: N/A
Revisiting Rating Scale Development for Rater-Mediated Language Performance Assessments: Modelling Construct and Contextual Choices Made by Scale Developers
Knoch, Ute; Deygers, Bart; Khamboonruang, Apichat
Language Testing, v38 n4 p602-626 Oct 2021
Rating scale development in the field of language assessment is often considered in dichotomous ways: It is assumed to be guided either by expert intuition or by drawing on performance data. Even though quite a few authors have argued that rating scale development is rarely so easily classifiable, this dyadic view has dominated language testing research for over a decade. In this paper we refine the dominant model of rating scale development by drawing on a corpus of 36 studies identified in a systematic review. We present a model showing the different sources of scale construct in the corpus. In the discussion, we argue that rating scale designers, just like test developers more broadly, need to start by determining the purpose of the test, the relevant policies that guide test development and score use, and the intended score use when considering the design choices available to them. These include considering the impact of such sources on the generalizability of the scores, the precision of the post-test predictions that can be made about test takers' future performances and scoring reliability. The most important contributions of the model are that it gives rating scale developers a framework to consider prior to starting scale development and validation activities.
Descriptors: Rating Scales, Test Construction, Language Tests, Test Use, Scores, Applied Linguistics, Evaluation Research, Test Reliability
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://bibliotheek.ehb.be:2814
Publication Type: Journal Articles; Reports - Evaluative; Information Analyses
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A