The New General Service List Test (NGSLT) (Stoeckel & Bennett, 2015) was designed as a diagnostic test to measure students' written receptive vocabulary knowledge. This test battery was developed based upon the New General Service List (NGSL) (Browne, 2013), which makes it appealing to teachers in Japan, and especially those who see vocabulary…

Testing learners' English proficiency is central to university English classes in Japan. This study developed and implemented a set of parallel online receptive aural and visual vocabulary tests that would predict learners' English proficiency. The tests shared the same target words and choices--the main difference was the presentation of the…

Assessing speaking is one of the most difficult tasks in computer based language testing. Many countries all over the world face the need to implement standardized language tests where speaking tasks are commonly included. However, a number of problems make them rather impractical such as the costs, the personnel involved, the length of time for…

The vast majority of second language teachers feels confident about their instructional performance and does not usually have much difficulty with their teaching thanks to their professional training and accumulated classroom experience. Nonetheless, many second language teachers may not have received sufficient training in test development to…

In alignment studies, expert raters evaluate assessment items against standards and ratings are used to compute various alignment indices. Questions about rater reliability, however, are often ignored or inadequately addressed. This paper reports the results of a generalizability theory study of cognitive demand and rigor ratings of assessment…

In this speech, the author covers a lot of ground. In the first half of his speech, the author gives a brief summary of the last 40 years of the history of language testing, from his perspective. The author reviews these years more or less by decade. Additionally, he discusses the evolution of the profession of language testing during this period,…

A study examined inter-rater reliability on the American Council on the Teaching of Foreign Languages/Educational Testing Service (ACTFL/ETS) oral language proficiency rating scale. Seven raters, all elementary or intermediate college Spanish teachers given only brief formal training in the use of the scale, evaluated recorded interviews with…

This paper outlines the development of a performance assessment measure of language speaking ability, the Language Ability Assessment System (LAAS), which is highly reliable and can be examined for reliability through modern measurement theories, such as generalizability theory (G-theory) and the many-facet Rasch theory. LAAS was developed to…

A preliminary study is reported of the use of new multifaceted Rasch measurement mechanisms for investigating rater characteristics in language testing. Ratings from four judges of scripts from 50 candidates taking the International English Language Testing System test, a test of English for Academic Purposes, are analyzed. The analysis…

Results of the International English Language Testing System (IELTS) battery trials in Australia are reported. The IELTS tests of productive language skills use direct assessment strategies and subjective scoring according to detailed guidelines. The receptive skills tests use indirect assessment strategies and clerical scoring procedures.…

To determine how many occurrences of a syntactic structure are necessary to provide sampling reliability, two one-half hour spontaneous language samples were elicited from each of 15 language disordered students (5 to 6 years old). Sessions were divided into two periods, one for the child's telling about pictures and toys and the other for…

Although the standard-setting method of W. H. Angoff (1971) has broad-based support in the research literature, inconsistencies in the resulting standards do occur. Sources of these inconsistencies are examined in a study of judges, competencies (items), rounds (replications), and the interactions among them. A modified Angoff approach was used to…

Reports on study in which students of Hebrew as a second language took four versions of oral proficiency test. Results indicate that different speech styles and topics significantly affected students' scores, and correlational analyses between pairs pointed to low reliability and lack of stability of the test. Urges caution in making decisions…

This study investigates the Test of English as a Foreign Language (TOEFL), in particular the relative contributions to score dependability (analogous to classical theory reliability) of various numbers of items and subtests as well as the decision dependability at different cut points. Research questions that apply to the overall TOEFL battery and…

A discussion of the differences between the Test of English as a Foreign Language (TOEFL), an American test battery, and the Cambridge English Examinations (Cambridge), a British battery, focuses on the different approaches to language test development embodied in the tests as the source of difficulty in translating between them for individual…