One of the most critical steps in the test development process is defining the construct, or the knowledge, skills, or abilities, to be assessed. This foundational step provides the basis for initial assumptions about the meaning of test scores and serves as a reference for subsequent validity research. In this paper, we describe the purpose of…

A key piece of a validity argument for a language assessment tool is clear overlap between assessment tasks and the target language use (TLU) domain (i.e., the domain description inference). The TOEFL 2000 Spoken and Written Academic Language (T2K-SWAL) corpus, which represents a variety of academic registers and disciplines in traditional…

The broad range of English language teaching and learning contexts present in the world today necessitates high quality assessment instruments that can provide reliable and meaningful information about learners' English proficiency levels to relevant stakeholders. The "TOEFL Junior"® tests were recently introduced by Educational Testing…

This study is an evaluation of the performance of automated speech scoring for speakers with documented or suspected speech impairments. Given that the use of automated scoring of open-ended spoken responses is relatively nascent and there is little research to date that includes test takers with disabilities, this small exploratory study focuses…

In this study, we assessed the invariance in the factor structure underlying English-language proficiency for two groups of adolescent learners in Japan: students in middle school (ages 13-15 years) and students in high school (ages 16-18 years). Language proficiency was measured using the "TOEFL Junior"® Comprehensive test, an…

This report is the fifth in a series concerning English language proficiency (ELP) assessments for English learners (ELs) in kindergarten through 12th grade in the United States. The series, produced by Educational Testing Service (ETS), is intended to provide theory and evidence-based principles and recommendations for improving next-generation…

This paper is the second in a series from Educational Testing Service (ETS) that conceptualizes next-generation English language proficiency (ELP) assessment systems for K-12 English learners (ELs) in the United States.The first paper articulated a high-level conceptualization of next-generation ELP assessment systems (Hauck, Wolf, & Mislevy,…

In this study, we developed assistive tools and resources to support TOEIC® Listening test item generation. There has recently been an increased need for a large pool of items for these tests. This need has, in turn, inspired efforts to increase the efficiency of item generation while maintaining the quality of the created items. We aimed to…

The purpose of this report is to propose a theory of action for the development of an alternate English language proficiency assessment (AELPA) system to support the integrated instruction and assessment for English learners with significant cognitive disabilities (ELSCDs). This theory of action examines the purposes of an ELP assessment system…

The past decade has seen an emerging interest in mapping (aligning or linking) test scores to language proficiency levels of external performance scales or frameworks, such as the Common European Framework of Reference (CEFR), as well as locally developed frameworks, such as China's Standards of English Language Ability (CSE). Such alignment is…

This project examined academic reading in two contrasting English as a medium of instruction (EMI) university settings in Nepal and Sweden and the unique challenges facing students who are studying in a language other than their primary language. The motivation for the project was to explore the role of high-stakes testing in EMI contexts and the…

This study uses online surveys and phone interviews with Taiwanese college students to investigate their perceptions of using the "TOEIC"® test scores to meet an English-language graduation requirement. Results indicate that students have positive views about the use of the TOEIC test scores for graduation and believe that preparing to…

This report explores the ways in which human resource (HR) managers use "TOEIC"® scores to inform hiring, promotion, and training decisions in an international workplace. Two data sources were used (a) previously collected test users' testimonials that described managers' use of TOEIC scores to inform HR decisions and (b) test-use…

A model-based approach for matching language learners to texts of appropriate difficulty is described. Results are communicated to test takers via a targeted reading range expressed on the reporting scale of an automated text complexity measurement tool (ATCMT). Test takers can use this feedback to select reading materials that are well matched to…

The purpose of the present study is to examine whether performance on the "TOEFL iBT"® Reading practice test is affected by 3 different levels of feedback provided to learners upon completion of reading exercises: (a) correctness of learner response (the knowledge of correct results [KCR] feedback), (b) KCR feedback and rationales for…