Three methods based on the nonparametric item response theory (IRT) of R. J. Mokken for the estimation of the reliability of single dichotomous test items are discussed. Analytical and Monte Carlo studies show that one method, designated "MS," is superior because of smaller bias and smaller sampling variance. (SLD)

Developed a procedure for defining achievement levels on continuous scales using aspects of Guttman scaling (L. Guttman, 1950) and Item Response Theory. Using data from high school mathematics tests for about 6,000 students, found the new procedure to have higher reliability, higher classification consistency, and lower classification error than…

Empirically weighted scores for Rest's Defining Issues Test were found to be more reliable than the simple sum of scores theoretically weighted sum, or Rest's p scores. They also had slightly higher correlations with Kohlberg's interview scores. Empirically weighted scores also showed more significant change in two longitudinal studies. (CTM)

Binary, probability, and ordinal scoring procedures for multiple-choice items were examined. In two situations, it was found that both the probability and ordinal scoring systems were more reliable than the binary scoring method. (Author/CTM)

Three studies are described in which choice reaction time (RT) was related to such psychometric ability measures as verbal comprehension, numerical reasoning, hidden figures, and progressive matrices tests. Fairly consistent negative correlations were found between these tests and choice RT when high school samples were used. (Author/CTM)

This special issue covers recent technical developments in the field of criterion-referenced testing. An introduction, six papers, and two commentaries dealing with test development, test score uses, and evaluation of scores review relevant literature, offer new models and/or results, and suggest directions for additional research. (SLD)

A set of Tests of Scientific Thinking were developed for possible use as criterion measures in research on creativity. Scores on the tests describe both quality and quantity of ideas produced in formulating hypotheses, evaluating proposals, solving methodological problems, and devising methods for measuring constructs. (Author/CTM)

Five studies are presented describing the development of a self-report inventory for measuring individual differences in learning processes. Factor analysis of items yielded four scales: Synthesis-Analysis, Study Methods, Fact Retention, and Elaborative Processing. There were no sex differences, and the scales demonstrated acceptable reliabilities…

A comparison of the relative ordering power of separate and grouped-items true-false tests indicated that neither type of test was uniformly superior to the other across all levels of knowledge of examinees. Grouped-item tests were found superior for examinees with low levels of knowledge. (Author/CTM)

Two methods--non-distributional and normal--are derived for estimating measures of pass-fail reliability. Both are based on the Spearman Brown formula and require only a single test administration. Results from a simulation (n=20,000 examinees) and a licensure examination (n=4,828 examinees) illustrate these methods. (SLD)

Derivations of several item selection algorithms for use in fitting test items to target information functions (IFs) are described. These algorithms, which use an average growth approximation of target IFs, were tested by generating six test forms and were found to provide reliable fit. (SLD)

Textbook calculations of statistical power or sample size follow from formulas that assume that the variables under consideration are measured without error. However, in the real world of behavioral research, errors of measurement cannot be neglected. The determination of sample size is discussed, and an example illustrates blocking strategy.…

Formulas are provided for estimating the reliability of a linear composite of non-equivalent subtests given the reliabilities of component subtests. The reliability of the composite is compared to that of its components. An empirical example uses data from 170 children aged 4 through 8 years performing 34 Piagetian tasks. (SLD)

Reliability and validity of 4-point and 6-point scales were assessed using a new model-based approach to fit empirical data from 165 graduate students completing an attitude measure. Results suggest that the issue of four- versus six-point scales may depend on the empirical setting. (SLD)

A method is proposed for sequential evaluation of reliability of psychometric instruments. Sample size is unfixed; a test statistic is computed after each person is sampled and a decision is made in each stage of the sampling process. Results from a series of Monte-Carlo experiments establish the method's efficiency. (SLD)