This simulation study sought to compare four different computerized adaptive testing (CAT) content-balancing procedures designed for use in a multidimensional assessment with respect to measurement precision, symptom severity classification, validity of clinical diagnostic recommendations, and sensitivity to atypical responding. The four…

Although there is an increasing number of online tests in the world, little research is currently known in Spain today. Assessment has become an integral part of education and the implications of the various uses of language testing go beyond the educational settings (Douglas, 2010; Fulcher, 2010). This study describes the PAULEX project. This…

A computer adaptive test (CAT) is a delivery methodology that serves the larger goals of the assessment system in which it is embedded. A thorough analysis of the assessment system for which a CAT is being designed is critical to ensure that the delivery platform is appropriate and addresses all relevant complexities. As such, a CAT engine must be…

Many challenges exist for high-stakes testing programs offering continuous computerized administration. The automated assembly of test questions to exactly meet content and other requirements, provide uniformity, and control item exposure can be modeled and solved by mixed-integer programming (MIP) methods. A case study of the computerized…

In cognitive diagnosis a Q-matrix (Tatsuoka, 1983, 1990), which is an incidence matrix that defines the relationships between test items and constructs of interest, has great impact on the nature of performance feedback that can be provided to score users. The purpose of the present study was to identify meaningful skill coding categories that…

Psychometricians continue to introduce new approaches to setting cut scores for educational assessments in an attempt to improve on current methods. In this paper we describe the Item-Descriptor (ID) Matching method, a method based on IRT item mapping. In ID Matching, test content area experts match items (i.e., their judgments about the knowledge…

The second national sample assessment of Information and Communication Technology (ICT) literacy at Year 6 and Year 10 will occur in 2008. The assessment happens every three years and the first occurred in 2005. This document provides information about the ICT literacy assessment including: (1) Education Ministers' decisions regarding ICT; (2) the…

The information and assessment materials in these resources have been designed to assist teachers to gauge their own students' proficiency in Information and Communication Technologies (ICT) literacy. By examining modules from the National Year 6 and Year 10 ICT Literacy Assessment teachers may be able to design similar tasks and to judge their…

Students with disabilities and special needs have faced challenges in accessing educational content, and in taking traditional pen-and-paper tests. How might technology improve the process, while making statewide tests truly accessible to all students? NimbleTools is the first computer-based test delivery system that incorporates principles of…

Traditionally, the fixed-length linear paper-and-pencil (P&P) mode of administration has been the standard method of test delivery. With the advancement of technology, however, the popularity of administering tests using adaptive methods like computerized adaptive testing (CAT) and multistage testing (MST) has grown in the field of measurement…

Examined the effectiveness of the Sympson-Hetter technique and rotated content balancing relative to no exposure control and no content rotation conditions in a computerized adaptive testing system based on the partial credit model. Simulation results show the Sympson-Hetter technique can be used with minimal impact on measurement precision,…

Item selection methods in computerized adaptive testing (CAT) can yield extremely skewed item exposure distribution in which items with high "a" values may be over-exposed while those with low "a" values may never be selected. H. Chang and Z. Ying (1999) proposed the a-stratified design (ASTR) that attempts to equalize item…

Studied three stratification designs for computerized adaptive testing in conjunction with three well-developed content balancing methods. Simulation study results show substantial differences in item overlap rate and pool utilization among different methods. Recommends an optimal combination of stratification design and content balancing method.…

The multistage alpha-stratified computerized adaptive testing (CAT) design advocated a new philosophy of pool management and item selection using low discriminating items first. It has been demonstrated through simulation studies to be effective both in reducing item overlap rate and enhancing pool utilization with certain pool types. Based on…

This simulation investigated two procedures that reduce differences between paper-and-pencil testing and computerized adaptive testing (CAT) by making CAT content sensitive. Results indicate that the price in terms of additional test items of using constrained CAT for content balancing is much smaller than that of using testlets. (SLD)