Given the lack of computational tools available for assessing second language (L2) production in Korean, this study introduces a novel automated tool called the Korean Syntactic Complexity Analyzer (KOSCA) for measuring syntactic complexity in L2 Korean production. As an open-source graphic user interface (GUI) developed in Python, KOSCA provides…

With the present study I investigated the sources of score variance and dependability in a local oral English proficiency test for potential international teaching assistants (ITAs) across four first language (L1) groups, and suggested alternative test designs. Using generalizability theory, I examined the relative importance of L1s (i.e., Indian,…

This study investigated the effects of working memory capacity (WMC), first language (L1) syllogistic inferencing ability, and second-language (L2) linguistic knowledge on L2 listening comprehension for passages of different lengths. Participants were 193 Korean ninth-grade learners of English. A path analysis was used to examine multivariate…

The present study examined the reliability of the reading, listening, speaking, and writing section scores for the TOEFL iBT® test and their interrelationship in order to collect empirical evidence to support, respectively, the "generalization" inference and the "explanation" inference in the TOEFL iBT validity argument…

This study investigated the construct validity of a local speaking test for international teaching assistants (ITAs) from a fairness perspective, by employing a multi-group confirmatory factor analysis (CFA) to examine the impact of task type and examinee first language (L1) background on the internal structure of the test. The test consists of…

Based on evidence that listeners may favor certain foreign accents over others (Gass & Varonis, 1984; Major, Fitzmaurice, Bunta, & Balasubramanian, 2002; Tauroza & Luk, 1997) and that language-test raters may better comprehend and/or rate the speech of test takers whose native languages (L1s) are more familiar on some level (Carey,…

We adapted three practice College Scholastic Ability Tests (CSAT) of English listening, each with five-option items, to create four- and three-option versions by asking 73 Korean speakers or learners of English to eliminate the least plausible options in two rounds. Two hundred and sixty-four Korean high school English-language learners formed…

Despite the importance of having a reliable and valid measure of Second Language (L2) proficiency, L2 researchers of less commonly taught languages rarely have such a tool. Existing proficiency measures (e.g., DLPT, OPI) are often costly, labor-intensive, time-consuming, or unavailable to the public. With the intent to provide a practical and…

This study investigated the validity of four theoretically motivated traits of writing ability across English and Korean, based on elementary school students' responses to letter- and story-writing tasks. Their responses were scored analytically and analyzed using confirmatory factor analysis. The findings include the following. A model of writing…

Language performance assessments typically require human raters, introducing possible error. In international examinations of English proficiency, rater language background is an especially salient factor that needs to be considered. The existence of rater language background-related bias in writing performance assessment is the object of this…

This study used a mixed methods research approach to examine how native English-speaking (NS) and non-native English-speaking (NNS) teachers assess students' oral English performance. The evaluation behaviors of two groups of teachers (12 Canadian NS teachers and 12 Korean NNS teachers) were compared with regard to internal consistency, severity,…

This study applied generalizability theory to investigate the contributions of persons, items, sections, and language backgrounds to the score dependability of the Test of English for International Communication (TOEIC). I replicated and extended Brown's (1999) study of the Test of English as a Foreign Language (TOEFL), using data from two…

A study investigated the factorial distinctiveness of two receptive language skills, listening and reading, and the equivalence of factor structure across two groups. Subjects were 156 students, Korean-Americans and non-Korean-Americans, in grades two through four in a two-way bilingual education program. Students were tested in listening and…