Presents simplified methods for deriving estimates of the consistency of criterion-referenced, English-as-a-Second-Language tests, including (1) the threshold loss agreement approach using agreement or kappa coefficients, (2) the squared-error loss agreement approach using the phi(lambda) dependability approach, and (3) the domain score…

Suggests that performance tests as currently represented in the American Council on the Teaching of Foreign Languages (ACTFL)-Oral Proficiency Interview (OPI) may not adequately address the basic concerns brought about by the perceived shortcomings of academic second language programs. Supports this argument with a critical analysis of the ACTFL…

Investigates differential item functioning (DIF) across two different broad language groupings, Asian and European, in a speaking test in which the test takers' responses were rated polytomously. Data were collected from 1038 nonnative speakers of English from France, Hong Kong, Japan, Spain, Switzerland, and Thailand who took the SPEAK test in…

Reports on the potential problems in scoring responses to sequencing tests, the development of a computer program to overcome these difficulties, and an exploration of the value of scoring procedures. (Author/VWL)

Explores the effect on pair-task performance of test-takers' familiarity with their partner. Japanese learners performed a series of tasks, once with a friend and again with a person who was not familiar to them. Results support findings in the literature that suggest learners vary their language when interacting with familiar or unfamiliar…

The present study investigated the effects of reducing the number of options per item on psychometric characteristics of a Japanese EFL university entrance examination. A four-option multiple-choice reading test used for entrance screening at a university in Japan was later converted to a three-option version by eliminating the least frequently…

An English-as-a-second-language reading comprehension test is analyzed, first using classical measurement procedures and then using the Rasch Model latent trait procedures. Results indicate that the Rasch Model analysis procedure produced numerous advantages for the test developer. Summarizes and describes 13 of these advantages. (SED)

Describes a three-phase process model for interface design, drawing on practices developed in the software industry and adapting them for computer-based languages tests. Describes good practice in initial design, emphasizes the importance of usability testing, and argues that only through following a principled approach to interface design can the…

Discusses theoretical and practical issues pertaining to translation or adaptation of educational assessments in the United States. These include the role of language proficiency and academic background in performance on standards-based achievement tests in different languages, factors affecting the decision to translate tests in different…

Explores the role of task and task-based assessment in a collegiate foreign language department that shifted its entire undergraduate curriculum from a form-based normative approach to a language-use and language-meaning orientation for instruction. Examines how demands for specificity that characterize task-based assessment contributed…

Studies the application of the letter-deletion procedure (LDP), which matches the advantages of both rational deletion and C-Tests and avoids their disadvantages to English-as-a-Second-Language tests. LDP flexibly reduces redundancy in a text by deleting letters in item words according to specified principles. (Author/LMO)

This study addressed the question of the consistency of rater characteristics over time, and investigated the potential of multifaceted Rasch measurement in exploring this question. The purpose of the research is to investigate the use of these analytical techniques in rater training for the speaking subtest of the Occupational English Test, an…

Responds to a previous article in which the author of this article's work on reading subskills is critiqued. The author states that, despite the criticism of his work, he feels the critique offers a useful contribution to the debate about subskills and judgments. (JL)

In an examination of definition of oral proficiency, this study investigated the phenomenon of variation in questions posed by interviewers at key junctures in the interview process. It is argued that perceptions of oral proficiency are reflected in the extent of accommodation in interviewer questioning. (23 references) (Author/LB)

Explored the relative contributions to Test of English as a Foreign Language (TOEFL) score dependability of various numbers of persons, items, subtests, languages, and their various interactions. Sampled 15,000 test takers, 1000 each from 15 different language backgrounds. (Author/VWL)