This study investigates how different form-focused instruction (FFI) timing impacts English as a foreign language (EFL) learners' grammar development. A total of 169 Chinese middle school learners were assigned to four conditions randomly: control, before-isolated FFI, integrated FFI, and after-isolated FFI. The three experimental groups received…

This report describes the development of efficient second language (L2) writing assessment tasks designed specifically for low-proficiency learners of English to be included in the "TOEFL® Essentials"™ test. Based on the can-do descriptors of the Common European Framework of Reference for Languages for the A1 through B1 levels of…

This study aimed to compare selection patterns of distractors (incorrect options) according to test taker proficiency regarding Japanese students' summarization skills of an English paragraph. Participants included 414 undergraduate students, and the test comprised three summarization process types--deletion, generalization, and integration.…

One of the most interesting studies on the role of L1 and contrastive analysis in vocabulary teaching is by Laufer and Girsai (2008). However, due to some methodological issues, their research findings are open to criticism and controversy. The current study aimed to replicate the research with a more rigorous design to re-investigate the…

Background: High-stakes tests have an immense washback effect on what students learn and affect the content of student learning. However, if students fail to recognize the abilities that the test developers intend to measure, they are less likely to learn what the test developers wish them to learn. This study aims to investigate test-taker…

The purpose of this study is to explore patterns in model-data fit related to subgroups of test takers from a large-scale writing assessment. Using data from the SAT, a calibration group was randomly selected to represent test takers who reported that English was their best language from the total population of test takers (N = 322,011). A…

In this research, we investigated the suitability of implementing "e-rater"® automated essay scoring in a high-stakes large-scale English language testing program. We examined the effectiveness of generic scoring and 2 variants of prompt-based scoring approaches. Effectiveness was evaluated on a number of dimensions, including agreement…

Title III of Public Law 107-110 (No Child Left Behind; NCLB) provided for creation of assessments of English language learners (ELLs) and established, through the Enhanced Assessment Grant program, a platform from which four consortia of states developed ELL tests aligned to rigorous statewide content standards. Those four tests (ACCESS for ELLs,…

This paper reports the results of generalizability theory (G) analyses done for new writing and speaking tasks for the Test of English as a Foreign Language (TOEFL). For writing, a special focus was placed on evaluating the impact on the reliability of the number of raters (or ratings) per essay (one or two) and the number of tasks (one, two, or…

Presented at the Annual National Council on Measurement in Education (NCME) in New York in March 2008. This presentation explores how different item weighting can affect the effective weights, validity coefficents and test reliability of composite scores among test takers.

Statistical and measurement properties are examined for features used in essay assessment to determine the generalizability of the features across populations, prompts, and individuals. Data are employed from TOEFL® and GMAT® examinations and from writing for Criterion?.

Three polytomous DIF detection techniques--the Mantel test, logistic regression, and polySTAND--were used to identify GRE® Analytical Writing prompts ("Issue" and "Argument") that are differentially difficult for (a) female test takers; (b) African American, Asian, and Hispanic test takers; and (c) test takers whose strongest…