An approximate statistical test of the equality of two intraclass reliability coefficients based on the same sample of people is derived. Such a test is needed when a researcher wishes to compare the reliability of two measurement procedures, and both procedures can be applied to results from the same group. (SLD)

Describes a structural equation model that permits estimation of the reliability index and coefficient of a composite index for congeneric measures. The method is also helpful in exploring the factorial structure of an item set, and its use in scale reliability estimation and development is illustrated. (SLD)

The dependence of reliability indices for mastery tests on mean and cutoff scores was examined in the case of three decision-theoretic indices. Dependence of kappa on mean and cutoff scores was opposite to that of the proportion of correct decisions, which was linearly related to average threshold loss. (Author/BW)

Dynamic generalizations of reliability and validity that will incorporate longitudinal or developmental models, using latent curve analysis, are discussed. A latent curve model formulated to depict change is incorporated into the classical definitions of reliability and validity. The approach is illustrated with sociological and psychological…

The clarification provided by Williams and Zimmerman on the reliability of gain scores is translated into recognizable patterns of change that tend to produce reliable or unreliable gain scores. The relevance of the traditional idea of reliability to the measurement of change is also discussed. (SLD)

Presents minimization and maximization models for parallel test construction under constraints. The minimization model constructs weakly and strongly parallel tests of minimum length, while the maximization model constructs weakly and strongly parallel tests with maximum test reliability. (Author/SLD)

Two sources of inconsistency were separated by reanalyzing data from a major study on short-term consistency. Little evidence was found for generalizability or behavioral predictability. Results supported the assumption that measurement error from short-term fluctuations is not due to systematic individual differences in response consistency.…

Two new methods for constructing a credibility interval (CI)--an interval containing a specified proportion of true validity description--are discussed, from a frequentist perspective. Tolerance intervals, unlike the current method of constructing the CI, have performance characteristics across repeated applications and may be useful in validity…

Generalizability theory is used to characterize and quantify expected variance in cutting scores and to compare the Nedelsky and Angoff procedures for establishing a cutting score. Results suggest that the restricted nature of the Nedelsky (inferred) probability scale may limit its applicability in certain contexts. (Author/BW)

Modified equations for the validity and reliability of difference scores that describe applied testing situations are examined. This examination reveals that simple gain scores can be more useful in research than has commonly been believed. Simple gain scores are neither inherently unreliable nor lack predictive validity. (SLD)

Issues arising from difference scores with zero reliability that nevertheless allow a powerful test of change are discussed. Issues include the appropriateness of underlying statistical models for psychological data and the relationship between difference scores and power. Increases in reliability always increase power for a fixed effect size.…

The unconditional--simultaneous--maximum likelihood (UML) estimation procedure for the one-parameter logistic model produces biased estimators. The UML method is inconsistent and is not a good alternative to conditional maximum likelihood method, at least with small numbers of items. The minimum Chi-square estimation procedure produces unbiased…

Proposes a method for obtaining standard errors and confidence intervals of composite reliability coefficients based on bootstrap methods and using a structural-equation-modeling framework for estimating the composite reliability of congeneric measures (T. Raykov, 1997). Demonstrates the approach with simulated data. (SLD)

Reviews relevant aspects of generalizability theory related to performance assessments and discusses the role of various facets in assessing the generalizability of performance assessments. Also considers some popular estimates of reliability for performance assessments from the perspective of generalizability theory. (SLD)

A multiple regression approach is used to assess the feasibility of reciprocal prediction between the Sixteen Personality Factor Questionnaire scales and the California Psychological Inventory scales (i.e., the prediction of each 16PF scale from the CPI scales and of each CPI scale from the 16PF scales). (RC)