In some popular test designs (including computerized adaptive testing and multistage testing), many item pairs are not administered to any test takers, which may result in some complications during dimensionality analyses. In this paper, a modified DETECT index is proposed in order to perform dimensionality analyses for response data from such…

As pointed out by Sijtsma ("in press"), coefficient alpha is inappropriate as a single summary of the internal consistency of a composite score. Better estimators of internal consistency are available. In addition to those mentioned by Sijtsma, an old dimension-free coefficient and structural equation model based coefficients are…

Social scientists are frequently interested in assessing the qualities of social settings such as classrooms, schools, neighborhoods, or day care centers. The most common procedure requires observers to rate social interactions within these settings on multiple items and then to combine the item responses to obtain a summary measure of setting…

The critical reactions of Bentler (2009, doi: 10.1007/s11336-008-9100-1), Green and Yang (2009a, doi: 10.1007/s11336-008-9098-4 ; 2009b, doi: 10.1007/s11336-008-9099-3), and Revelle and Zinbarg (2009, doi: 10.1007/s11336-008-9102-z) to Sijtsma's (2009, doi: 10.1007/s11336-008-9101-0) paper on Cronbach's alpha are addressed. The dissemination of…

Reliability captures the influence of error on a measurement and, in the classical setting, is defined as one minus the ratio of the error variance to the total variance. Laenen, Alonso, and Molenberghs ("Psychometrika" 73:443-448, 2007) proposed an axiomatic definition of reliability and introduced the R[subscript T] coefficient, a measure of…

We make theoretical comparisons among five coefficients--Cronbach's [alpha], Revelle's [beta], McDonald's [omega][sub h], and two alternative conceptualizations of reliability. Though many end users and psychometricians alike may not distinguish among these five coefficients, we demonstrate formally their nonequivalence. Specifically, whereas…

The major goal of this paper was to provide a definition and measure of profile reliability and considered the problem of constructing reliable profile dimensions. (Author/RK)

The rater agreement literature is complicated by the fact that it must accommodate at least two different properties of rating data: the number of raters (two versus more than two) and the rating scale level (nominal versus metric). While kappa statistics are most widely used for nominal scales, intraclass correlation coefficients have been…

To assess the reliability of congeneric tests, specifically designed reliability measures have been proposed. This paper emphasizes that such measures rely on a unidimensionality hypothesis, which can neither be confirmed nor rejected when there are only three test parts, and will invariably be rejected when there are more than three test parts.…

Two lower bounds to reliability in classical test theory, Guttman's lamda and Cronbach's alpha, are shown to be terms of an infinite series of lower bounds. All terms of this series are equal to reliability if and only if the test contains items which are tau-equivalent. (Author/JKS)

A coefficient derived from communalities of test parts represents greatest lower bound to Guttman's "immediate retest reliability." Constrained minimum trace factor analysis allows a consistent estimate of the greatest defensible internal-consistency coefficient. In modest size samples, this analysis capitalizes on chance, suggesting an…

The problem of scaling ordinal categorical data observed over two or more sets of categories measuring a single characteristic is addressed. Scaling is obtained by solving a constrained entropy model. A Kullback-Leibler statistic is generated that operationalizes a measure for the strength of consistency among the sets of categories. (TJH)

The relationships among Rao's canonical factor analysis, Kaiser and Coffrey's alpha factor analysis, and Cronbach's alpha coefficient are discussed. Implications for test reliability and validity are noted. (JKS)

Minimum trace factor analysis has been used to find the greatest lower bound to reliability. This technique, however, fails to be scale free. A solution to the scale problem is proposed through the maximization of the greatest lower bound as the function of weights. (Author/JKS)

The asymptotic standard deviation (SD) of the alpha coefficient with standardized variables is derived under normality. The research shows that the SD of the standardized alpha coefficient becomes smaller as the number of examinees and/or items increase. Furthermore, this research shows that the degree of the dependence of the SD on the number of…