Cognitive diagnosis models (CDMs) are psychometric models developed mainly to assess examinees' specific strengths and weaknesses in a set of skills or attributes within a domain. By adopting the Generalized-DINA model framework, the recently developed general modeling framework, we attempted to retrofit the PISA reading assessments, a…

Item analysis has essential roles in the learning assessment. The item analysis program is designed to measure student achievement and instructional effectiveness. This study was aimed to develop item-analysis program and verify its feasibility. This study uses a Research and Development (R & D) model. The procedure includes designing and…

Establishing cut scores using the Angoff method requires panelists to evaluate every item on a test and make a probability judgment. This can be time-consuming when there are large numbers of items on the test. Previous research using resampling studies suggest that it is possible to recommend stable Angoff-based cut score estimates using a…

Test-based accountability often produces score inflation. Most studies have evaluated inflation by comparing trends on a high-stakes test and a lower stakes audit test. However, Koretz and Beguin (2010) noted weaknesses of audit tests and suggested self-monitoring assessments (SMAs), which incorporate audit items into high-stakes tests. This…

A key issue of cognitive diagnostic models (CDMs) is the correct identification of Q-matrix which indicates the relationship between attributes and test items. Previous CDMs typically assumed a known Q-matrix provided by domain experts such as those who developed the questions. However, misspecifications of Q-matrix had been discovered in the past…

Pondering a math problem while she swings her sneakered feet from a chair, 12-year-old Andrea Guevara is helping researchers design an assessment that will shape the learning of 19 million students. The 8th grader, who came to the United States from Ecuador three years ago, is trying out two ways of providing English-language support on a…

This paper reports a feasibility study on developing computer model-based assessments of chemical reasoning at the high school level. Computer models are flash and NetLogo environments to make simultaneously available three domains in chemistry: macroscopic, submicroscopic, and symbolic. Students interact with computer models to answer assessment…

Problem Statement: Computerized adaptive testing (CAT) is a sophisticated and efficient way of delivering examinations. In CAT, items for each examinee are selected from an item bank based on the examinee's responses to the items. In this way, the difficulty level of the test is adjusted based on the examinee's ability level. Instead of…

This paper addresses the use of computer-based testing in distance education, based on the experience of Universitas Terbuka (UT), Indonesia. Computer-based testing has been developed at UT for reasons of meeting the specific needs of distance students as the following: (1) students' inability to sit for the scheduled test; (2) conflicting test…

This article provides practical advice on the development of test and item specifications. Because my experience has been with English as a second or foreign language, most examples are taken from this field, although they can be applied to other language tests. The article first discusses steps in the process of test development, including…

The results of a practical clinical examination in podiatric medicine administered to fourth-year students are presented. The examination could become the prototype of a Part III practical clinical examination under the auspices of the National Board of Podiatry Examiners. Its feasibility is established and problems and issues are discussed.…

A latent trait model is described that is appropriate for use with tests that measure more than one dimension, and its application to both real and simulated test data is demonstrated. Procedures for estimating the parameters of the model are presented. The research objectives are to determine whether the two-parameter logistic model more…

A multiple matching test--a 24-item Hebrew vocabulary test--was examined, in which distractors from several items are pooled into one list at the test's end. Construction of such tests was feasible. Reliability, validity, and reduction of random guessing were satisfactory when applied to data from 717 applicants to Israeli universities. (SLD)

The feasibility of pre-equating, or establishing conversions from raw to scaled scores through the use of pretest data before operationally administering a test, was investigated for the Scholastic Aptitude Test (SAT). Item-response theory based equating methods were used to estimate item parameters on SAT pretest data, instead of using final form…

A project examined the mathematics and science performance of vocational students in Ohio. Part I of the study achieved the first objective--to report the science and mathematics content contained within the Ohio Vocational Achievement Tests for the 38 program areas. Each of the 38 tests was presented and found to have sound psychometric…