The shortcomings of the conventional way of using and interpreting multiple-choice tests are summarized. Some theories and methodologies that can be applied for better use multiple-choice test items are described. Empirical facts are introduced to support the theoretical observations. New strategies are proposed that will reduce "noise"…

The construct of dispositions is well defined in national standards, and U.S. colleges of education are required to assess candidate dispositions to meet accreditation requirements. Measurement, however, is virtually non-existent. On-line reviews of college accreditation reports indicate that colleges are attempting to assess dispositions without…

The purpose of this study was to develop a psychometrically sound instrument to measure teachers' attitudes toward students with special needs, the Scale of Teacher's Attitudes Toward Inclusion (STATIC). Approximately 1,440 inservice teachers were asked to complete the STATIC. There were 516 respondents from 5 school districts in Alabama. Various…

A method is presented for using a restricted latent class model--a binary skills model--to determine skills required by a set of test items. Results from the method's application to reading achievement data for about 63,000 fourth graders indicate that the model offers useful information on test structure and examinee ability. (TJH)

Effects of altering test difficulty on examinee ability measures and test length in a computer adaptive test were studied for 225 medical technology students in 3 test difficulty conditions. Results suggest that, with an item pool of sufficient depth and breadth, acceptable targeting to test difficulty is possible. (SLD)

Effectiveness of the Mantel-Haenszel statistic in detecting differentially functioning test items was studied when the internal criterion was varied, using a dataset from a life skills examination taken by 1,000 Anglo-American and 1,000 Native American high school students. Choice of criterion variable has a substantial effect on detection. (SLD)

A procedure for interpreting multiple-discrimination indices from a multidimensional item-response theory analysis is described and demonstrated with responses of 1,635 high school students to a multiple-choice test. The procedure consists of converting discrimination parameter estimates to direction cosines and analyzing the angular distances…

This paper reports on the development and validation of the Assessment Profile: Research Version, an observation checklist containing 87 items that reflect developmentally appropriate practices in early childhood programs. The Assessment Profile provides quantitative measures of classroom environments and teaching practices which facilitate the…

Item response theory (IRT) has been used extensively to study differential item functioning (dif) and to identify potentially biased items. The use of IRT for diagnostic purposes is less prevalent and has received comparatively less attention. This study addressed differential objective function (dof) to identify potentially biased content units.…

A study was conducted to investigate the difference in student performance on multiple choice (MC) and constructed response (CR) items relative to the achievement levels of the National Assessment of Educational Progress (NAEP). The study included an investigation of how estimates of student performance were affected by item response theory (IRT)…

The effectiveness of two five-point Likert response scales with different verbal anchors was compared. Four anonymous written questionnaires addressing characteristics of effective schools were developed. Each questionnaire was prepared in two formats, one with each response scale. The 4 questionnaires were composed of between 140 and 200 items. A…

A scaling procedure is proposed, based on item response theory (IRT), to fit non-hierarchical test structure as well. The binary scores of a test of English were used for calculating the probabilities of answering each item correctly. The probability matrix was factor analyzed, and the difficulty intervals or estimates corresponding to the factors…

The sources of multidimensionality found in several different forms of a licensure examination were studied. The relationship between one source of multidimensionality, differential item functioning (DIF) (or factors producing DIF), and content characteristics was explored in an attempt to isolate aspects of training or curriculum that could…

Problems of validity and reliability of concept mapping are addressed by using item-response theory (IRT) models for scoring. In this study, the overall structure of students' concept maps are defined by the number of links, the number of hierarchies, the number of cross-links, and the number of examples. The study was conducted with 92 students…

A benefit of using the multifaceted Rasch model is the ability to factor out or control for confounding factors in the estimation of person ability and item difficulty. This study experiments with a variation of the multifaceted Rasch analysis in calibrating the effects of demographic characteristics that are intended to overcome the problem of…