The study evaluated a recently postponed national test policy on the use of external agencies' English tests measuring four skills for Japanese university admission purposes. Using Kunnan's principles of fairness and justice, we generated and evaluated three claims regarding the test policy: vocabulary coverage of the tests, the justifiability of…

The present study examined the reliability of the reading, listening, speaking, and writing section scores for the TOEFL iBT® test and their interrelationship in order to collect empirical evidence to support, respectively, the "generalization" inference and the "explanation" inference in the TOEFL iBT validity argument…

The purpose of the present study is to examine whether performance on the "TOEFL iBT"® Reading practice test is affected by 3 different levels of feedback provided to learners upon completion of reading exercises: (a) correctness of learner response (the knowledge of correct results [KCR] feedback), (b) KCR feedback and rationales for…

This small-scale qualitative study considers feedback and results reported for two major large-scale English language tests administered in Japan: the Global Test of English Communication for Students (GTECfS) and the Eiken Test in Practical English Proficiency (Eiken). Specifically, it examines current score-reporting practices in student and…

Background: This small-scale needs analysis study examined content course faculty members' perspectives on English language demands in their courses and their perceptions of the introduction of four-skill English language assessment for university entrance examinations currently being planned as part of a major English education reform in Japan.…

The present study examined the factor structures across features of 446 examinees' responses to a writing task that integrates reading and listening modalities as well as reading and listening comprehension items of the TOEFL iBT[R] (Internet-based test). Both human and automated scores obtained for the integrated essays were utilized. Based on a…

This article presents the authors' response on the two commentators' (Fred Davidson and Charles Alderson) reviews of the 2009 Special Issue of "Language Assessment Quarterly" on "Cognitive Diagnosis and Q-matrices in Language Assessment." The Special Issue represents several years of research work on cognitive diagnosis approaches (CDA) carried…

This study investigates the value of reporting the reading, listening, speaking, and writing section scores for the "TOEFL iBT"® test, focusing on 4 related aspects of the psychometric quality of the TOEFL iBT section scores: reliability of the section scores, dimensionality of the test, presence of distinct score profiles, and the…

A variety of cognitive diagnosis approaches (CDAs) have been used by researchers to extract fine-grained information about the linguistic knowledge states and skill mastery levels of test takers based on their performance on language tests. Although some of these approaches have been in use in the psychometric community for quite some time, many…

The present study investigated the functioning of three psychometric models for cognitive diagnosis--the general diagnostic model, the fusion model, and latent class analysis--when applied to large-scale English as a second language listening and reading comprehension assessments. Data used in this study were scored item responses and incidence…

The study investigated the criterion-related validity of the "Test of English as a Foreign Language"[TM] Internet-based test (TOEFL[R] iBT) Listening section by examining its relationship to a criterion measure designed to reflect language-use tasks that university students encounter in everyday academic life: listening to academic…

This construct validation study investigated the factor structure of the Test of English as a Foreign Language[TM] Internet-based test (TOEFL[R] iBT). An item-level confirmatory factor analysis was conducted for a test form completed by participants in a field study. A higher-order factor model was identified, with a higher-order general factor…

In cognitive diagnosis a Q-matrix (Tatsuoka, 1983, 1990), which is an incidence matrix that defines the relationships between test items and constructs of interest, has great impact on the nature of performance feedback that can be provided to score users. The purpose of the present study was to identify meaningful skill coding categories that…

The present study investigated the factor structure of a field trial sample of the Test of English as a Foreign Language™ Internet-based test (TOEFL® iBT). An item-level confirmatory factor analysis (CFA) was conducted for a polychoric correlation matrix of items on a test form completed by 2,720 participants in the 2003-2004 TOEFL iBT Field…

This is a construct validation study of a second language speaking assessment that reported a language profile based on analytic rating scales and a composite score. The study addressed three key issues: score dependability, convergent/discriminant validity of analytic rating scales and the weighting of analytic ratings in the composite score.…