In this quasi-experimental study, the effects of lexical coverage through pseudo word manipulation in dialogue comprehension are investigated. Forty-four first-year students in a Japanese university listened to five dialogues at different lexical coverage levels: 98%, 95%, 90%, 85%, and 83%. The results of the comprehension tests confirm the…

Distractors in tests are included to divert attention away from correct answers choices. Knowing what types of distractors commonly appear in tests will benefit test takers as they can prepare themselves beforehand. Therefore, this qualitative study was aimed at making assumptions on plausible lexical distractors which are expected to appear in…

The aim of this mixed methods study is to identify the underlying structure of the construct of Foreign Language Anxiety in integrated listening-to-speak tasks. First, the analysis of the qualitative interviews with six postsecondary ESL learners reveals that anxiety for integrated speaking stems from four different factors: "listening,"…

Evidence of comparability is generally needed whenever there are variations in the conditions of an assessment administration, including variations introduced by the administration of an assessment on multiple digital devices (e.g., tablet, laptop, desktop). This article is meant to provide a comprehensive examination of issues relevant to the…

In recent years, large-scale international assessments have been increasingly used to evaluate and compare the quality of education across regions and countries. However, measurement variance between different versions of these assessments often posts threats to the validity of such cross-cultural comparisons. In this study, we investigated the…

The purpose of test directions is to familiarize examinees with a test so that they respond to items in the manner intended. However, changes in educational measurement as well as the U.S. student population present new challenges to test directions and increase the impact that differential familiarity could have on the validity of test score…

Purpose: The purpose of this replication study was to determine the degree to which teachers and administrators in selected low-achieving Title I elementary schools: 1. Report that they teach the following test-taking skills to prepare their students to take the CST: establishing an appropriate student attitude toward testing, familiarity with the…

In this study, cross-cultural differences in cognitive test scores are hypothesized to depend on a test's cultural complexity (Cultural Complexity Hypothesis: CCH), here conceptualized as its content familiarity, rather than on its cognitive complexity (Spearman's Hypothesis: SH). The content familiarity of tests assessing short-term memory,…

Two studies compared the performance of Puerto Rican and African American Head Start children (total n=134) on familiar (description) and unfamiliar (one word labeling) test tasks. Results indicated children performed better on the familiar test task, which was also more sensitive in differentiating typically developing children from those with…

The effects of problem content on mathematics word problem performance were explored for 128 male and 128 female college students solving problems with masculine, feminine, and neutral (familiar and unfamiliar) cover stories. No effect of sex typing was found, and a small, but highly significant, effect was found for familiarity. (SLD)