Research Findings: This study examined the applicability of the Classroom Assessment Scoring System (CLASS) Pre-K (Pianta, La Paro, & Hamre, 2008) and its underpinning framework of teaching through interactions in typical Chinese kindergarten classrooms. A sample of 180 kindergarten classrooms in China was selected, and the CLASS was used to…

Provides a sample of seven published studies in different disciplines that inappropriately generalized reliability coefficients involving several raters to scores generated by a single rater. Score reliability when only one rater is used for scoring is lower than the score reliability for which two raters are used. (SLD)

It is erroneous to extend or generalize the inter-rater reliability coefficient estimated from only a (small) proportion of the sample to the rest of the sample data where only one rater is used for scoring, although such generalization is often made implicitly in practice. It is shown that if inter-rater reliability estimate from part of a sample…

Illustrates a number of ways in which confidence intervals for reliability coefficients can be estimated. Suggests that authors who submit articles to "Educational and Psychological Measurement" report confidence intervals for reliability estimates whenever they report score reliabilities and that they note the interval estimation…

Performed a meta-analysis of reliability estimates for Beck Depression Inventory (BDI) (A. Beck and others, 1961) scores across studies. Only 7.5% of studies reviewed (n=1,200) reported meaningful reliability estimates. Analysis results suggest that standard errors of measurement should be considered in addition to reliability estimates when…

Examined the effects of 2 examinee sample characteristics (group heterogeneity and performance level) on score reliability of optimal performance measurement using extant data sets of more 50,000 high school students and 10,000 sixth graders. Results suggest that both characteristics affect score reliability, and measurement error tends to be…