This paper addresses the intersection of testing and policy, situating test-driven impact and validation within the context of policy-led educational reform in Korea. I will briefly review the existing validation models. Then, arguing for an expansion of the conventional conceptualization of consequential validity research, I use Fairclough's…

This study investigated the effects of working memory capacity (WMC), first language (L1) syllogistic inferencing ability, and second-language (L2) linguistic knowledge on L2 listening comprehension for passages of different lengths. Participants were 193 Korean ninth-grade learners of English. A path analysis was used to examine multivariate…

This paper explores the underlying construct of both the English proficiency test for pilot and air traffic controller radiotelephony communication developed and administered in Korea and the ICAO language proficiency testing policy on which the test in Korea is based. It does so by canvassing the opinions of Korean airline pilots and air traffic…

This study assessed the factor structure of the Test of English for International Communication (TOEIC®) Listening and Reading test, and its invariance across subgroups of test-takers. The subgroups were defined by (a) gender, (b) age, (c) employment status, (d) time spent studying English, and (e) having lived in a country where English is the…

Typically, English language proficiency tests yield multiple scores--usually for each of the four traditional language domains. In order to maximize the usefulness of test scores, they may need to be accompanied by information concerning how they complement one another. Using self-assessments by some 2300 TOEIC test takers, this study aimed to…

Testlets are subsets of test items that are based on the same stimulus and are administered together. Tests that contain testlets are in widespread use in language testing, but they also share a fundamental problem: Items within a testlet are locally dependent with possibly adverse consequences for test score interpretation and use. Building on…

We adapted three practice College Scholastic Ability Tests (CSAT) of English listening, each with five-option items, to create four- and three-option versions by asking 73 Korean speakers or learners of English to eliminate the least plausible options in two rounds. Two hundred and sixty-four Korean high school English-language learners formed…

This study investigated factors that could affect inter-examiner reliability in the pronunciation assessment component of speaking tests. We hypothesized that the rating of pronunciation is susceptible to variation in assessment due to the amount of exposure examiners have to nonnative English accents. An inter-rater variability analysis was…

This study tracked gender differential item functioning (DIF) on the English subtest of the Korean College Scholastic Aptitude Test (KCSAT) over a nine-year period across three data points, using both the Mantel-Haenszel (MH) and item response theory likelihood ratio (IRT-LR) procedures. Further, the study identified two factors (i.e. reading…

Pictures are widely used to elicit expressive language skills, and pictures must be established as parallel before changes in ability can be demonstrated by assessment using pictures prompts. Why parallel prompts are required and what it is necessary to do to ensure that prompts are in fact parallel is not widely known. To date, evidence of…

The present study provides an overview of the impact of standardized EFL tests on EFL education in Korea. To achieve this goal, the paper (1) presents the status quo of EFL testing in the Korean context, (2) explores the nature of the EFL tests prevalent in the EFL testing market, and (3) investigates the overwhelming washback effects of EFL tests…

Discusses the Test for Creative Thinking--Drawing Production (TCT-DP), a creativity test suitable for most age and ability groups. Explores the possibility of implementing TCT-DP among kindergarten children in Korea. Results from a TCT-DP field study on 1366 Korean preschool children are comparable to those found in previously reported German…