Developed a statistical test for the hypothesis of the equality of extrapolated coefficients when the original values (Cronbach's alpha) are based on the same sample of persons and, therefore, are statistically dependent. Monte Carlo studies show that the test precisely controlled Type I error, even with small numbers of parttest units or raters.…

Contrary to common belief, reliability estimates of number-right multiple-choice tests are not inflated by speededness. Because examinees guess on questions when they run out of time, the responses to these questions generally show less consistency with the responses of other questions, and the reliability of the test will be decreased. The…

Large-scale testing programs involving classification decisions typically have multiple forms available and conduct equating to ensure cut-score comparability across forms. A test developer might be interested in the extent to which an examinee who happens to take a particular form would have a consistent classification decision if he or she had…

This note describes a procedure for estimating the range restriction component used in correcting correlations for unreliability and range restriction when an estimate of the reliability of a predictor is not readily available for the unrestricted sample. This procedure is illustrated with a few examples. (Contains 1 table.)

Developed a graphical method of determining adequate sample size based on the power of L. Feldt's (1969) test of the difference between two values of Cronbach's alpha coefficient. Discusses assumptions on which this approach is based. (SLD)

Studied the population discrepancy of coefficient alpha from the composite reliability coefficient for fixed congeneric measures with correlated errors and expressed it in terms of parameters of the measures. Recommends structural equation modeling for identifying cases in which the discrepancy can be large. (SLD)

Developed an approximate statistical test for the hypothesis of equality between the Spearman-Brown extrapolations of two independent values of Cronbach's alpha reliability coefficient. Monte Carlo simulations demonstrate that the procedure effectively controls Type I error. (SLD)

Gulliksen's matched random subtests method is a graphical method to split a test into parallel test halves, allowing maximization of coefficient alpha as a lower bound to the classical test reliability coefficient. This problem is formulated as a zero-one programing problem solvable by algorithms that already exist. (TJH)

Developed an index for assessing interrater agreement with respect to a single target using a multi-item rating scale. The variance of rater mean scale scores is used as the numerator of the agreement index. Studied four variants of a disattenuated agreement index that vary in the random response term used as the denominator. (SLD)

A longitudinal study of approximately 1,000 students aged 10-16 showed high stability of intelligence and creativity. Stability coefficients for intelligence were higher than those for creativity. Results supported the construct validity of creativity. (MH)

This article studies the ordinal reliability of (total) test scores. This study is based on a classical-type linear model of observed score (X), true score (T), and random error (E). Based on the idea of Kendall's tau-a coefficient, a measure of ordinal reliability for small-examinee populations is developed. This measure is extended to large…

The accuracy of the large sample standard error of weighted kappa appropriate to the non-null case was studied by computer simulation for the hypothesis that two independently derived estimates of weighted kappa are equal, and for setting confidence limits around a single value of weighted kappa. (Author/CTM)

Proposed a new asymptotic formula for estimating the sampling variance of a correlation coefficient corrected for unreliability and range restriction. A Monte Carlo simulation study of the new formula results in several positive conclusions about the new approach. (SLD)

Proposes a revised index of interrater agreement for multi-item ratings of a single target. This index is an inverse linear function of the ratio of the average obtained variance to the variance of the uniformly distributed random error. Discusses the importance of sample size for the index. (SLD)

Evaluated coefficient alpha under violations of two classical test theory assumptions: essential tau-equivalence and uncorrelated errors through simulation. Discusses the interactive effects of both violations with true and error scores. Provides empirical evidence of the derivation of M. Novick and C. Lewis (1993). (SLD)