An examination of the extent to which scores on the Stuttering Severity Instrument (SSI) for Children and Adults, Third Edition, accurately reflect 10 judges' observations of stuttering behaviors found that SSI scores obscured the wide range of judges' raw counts and did not accurately reflect the observational data from which they were derived.…

The behavior of each of 120 students in grades 9-12 was rated by 2 of the student's teachers using the Revised Behavior Problem Checklist. Results indicated a generally low to moderate degree of relationship among teacher ratings. It is recommended that clinicians collect behavioral ratings from many raters before reaching diagnostic conclusions.…

Forty-seven young children, 15 with disabilities, were observed 4 times for types and levels of engagement. Results indicated that engagement is difficult to measure through molecular data collection techniques because of error in dependability measures. The number of observed sessions could be increased to achieve generalizability, but increases…

The traditional kappa statistic in assessing interrater agreement is not adequate when multiraters and multiattributes are involved. In this article, latent trait models are proposed to assess the multirater multiattribute (MRMA) agreement. Data from the Third International Mathematics and Science Studies (TIMSS) are used to illustrate the…

A total of 1,487 eleventh grade students from the Hamburg (West Germany) school system were asked to complete four writing assignments used in an International Association for the Evaluation of Educational Achievement (IEA) study of writing assessment. In analyzing the writing samples, the study focused on: (1) between-rater effects; (2)…

This paper suggests that in addressing the issue of parent-professional congruence in child assessment, researchers should avoid focusing on the conventional aspects of interrater reliability and rater interchangeability, but rather should focus on the reliability of the pooled assessment information from parents and professionals. A…

This study tested the interrater reliability of the Modified Ashworth Scale in measuring upper and lower limb spasticity in 34 hemiplegic adult patients examined by 2 physiotherapists and 2 doctors. Findings indicated satisfactory reliability for upper limb spasticity but less satisfactory results for lower limb spasticity. (DB)

Multivalent relations, inferred as relationships with an added dimension of discernment, are realized as weighted graphs with multivalued edges. A unified treatment of the threshold problem is discussed and a reliability measure is produced to judge various partitions. (Author/EM)

Too many researchers speak of "the reliability of the test," thus indicating their basic misunderstanding of reliability. This paper explains classical reliability and the score features that influence coefficient alpha. It explains when coefficient alpha can be negative, even though it is conceptually a variance-accounted-for statistic.…

This paper describes the Measure of Epistemological Reflection (MER), an instrument to assess cognitive developmental level according to the Perry scheme of intellectual and ethical development. It contains sets of questions for each of the six cognitive domains: decision making, learner role, instructor role in the learning process, peer role in…

Development of the Stereotyped Behavior Scale for adolescents and adults with mental retardation is described. Use with 600 individuals resulted in refinement and a 26-item scale with an internal consistency alpha of 0.88, test-retest reliability of p=0.90, and interrater reliability of p=0.76. (DB)

The Quebec Adaptive Behavior Scale (QABS) is widely used in Quebec (Canada) to assess behavior of people with mental retardation in educational, vocational, residential or hospital settings. This study estimated the interrater agreement and test-retest reliability of the QABS. To determine test-retest reliability, the QABS was completed by 27…

Surveyed 65 couples beginning marital therapy and 37 couples from the community to examine interspousal reliability on reports of marital violence. Agreement between partners on the occurrence of violence was low to moderate for both samples. Husbands underreported their violent behavior, and/or wives tended to overreport violence performed by…

Marking reliability is purported to be produced by having an effective community of practice. No experimental research has been identified which attempts to verify empirically the aspects of a community of practice that have been observed to produce marking reliability. This research outlines what that community of practice might entail and…

The purpose of this paper is to compare the precision of direct and indirect measures of writing assessment using the test information functions from a graded response Item Response Theory (IRT) model. Subjects were 192 sophomore English students from a parochial high school in Wisconsin. Both direct and indirect measures of writing ability were…