ERIC Number: EJ1419029
Record Type: Journal
Publication Date: 2024
Pages: 30
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0265-5322
EISSN: EISSN-1477-0946
Speaking Performances, Stakeholder Perceptions, and Test Scores: Extrapolating from the Duolingo English Test to the University
Daniel R. Isbell; Dustin Crowther; Hitoshi Nishizawa
Language Testing, v41 n2 p233-262 2024
The extrapolation of test scores to a target domain - that is, association between test performances and relevant real-world outcomes - is critical to valid score interpretation and use. This study examined the relationship between Duolingo English Test (DET) speaking scores and university stakeholders' evaluation of DET speaking performances. A total of 190 university stakeholders (45 faculty members, 39 administrative staff, 53 graduate students, 53 undergraduate students) evaluated the comprehensibility (ease of understanding) and academic acceptability of 100 DET test-takers' speaking performances. Academic acceptability was judged based on speakers' suitability for communicative roles in the university context including undergraduate study, group work in courses, graduate study, and teaching. Analyses indicated a large correlation between aggregate measures of comprehensibility and acceptability (r = 0.98). Acceptability ratings varied according to role: acceptability for teaching was held to a notably higher standard than acceptability for undergraduate study. Stakeholder groups also differed in their ratings, with faculty tending to be more lenient in their ratings of comprehensibility and acceptability than undergraduate students and staff. Finally, both comprehensibility and acceptability measures correlated strongly with speakers' official DET scores and subscores (r [greater than or equal to] 0.74-0.89), providing some support for the extrapolation of DET scores to academic contexts.
Descriptors: Language Proficiency, Language Tests, Higher Education, Stakeholders, English for Academic Purposes, English Language Learners, Scores, Speech Skills, Speech Tests, Theory Practice Relationship, College Entrance Examinations, Test Validity
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://bibliotheek.ehb.be:2993
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A