This study, involving five experienced and trained speech language pathologists, categorized the elicited imitations of five profoundly and five severely prelingually hearing-impaired subjects using the Phonetic Level Evaluation. Failure to obtain acceptably high levels of reliability suggests that this measure may not yet be an accurate and…

Forty prototypical descriptions representing 4 aptitude domains and 4 talent fields were rated by 2,343 intermediate-level pupils and their teachers, and indices of interpeer agreement were computed. A majority of the prototypes maintained acceptable interpeer agreement levels. Interpeer agreement depended primarily on the specific aptitude or…

Transcript analysis is an important methodology to study asynchronous online educational discourse. The purpose of this study is to revisit reliability and validity issues associated with transcript analysis. The goal is to provide researchers with guidance in coding transcripts. For validity reasons, it is suggested that the first step is to…

One of the best methods of formative assessment in academic areas and a method that exemplifies the characteristics of good measures is Curriculum-Based Measurement (CBM; Deno, 1985). Developed at the University of Minnesota in the early 1970's, CBM has been researched in academic areas including mathematics computation, concepts, and…

The purpose of this paper is to discuss ways in which developmental psychology suffers from the lack of an appropriate technology of measurement and statistical analysis. The paper begins by noting that developmental psychology is the study of change; that individuals develop through a succession of "stages" which are separated by…

Community coalitions (CCs) have labored with some difficulty to demonstrate empirical evidence of effectiveness in preventing a wide range of adolescent problem behaviors. Training and technical assistance (TA) have been identified as important elements in promoting improved functioning of CCs. A reliable, valid, and inexpensive method to assess…

Cason and Cason's model of performance rating was used to determine the extent to which variation in reviewer standards affected the reliability and validity of the program review process used to select papers for inclusion in the annual program. Data analyzed were the overall recommendation for acceptance and ratings on seven quality criteria…

Generalizability theory, which subsumes classical measurement theory as a special case, provides a general model for estimating the reliability of observational rating data by estimating the variance components of the measurement design. Research data from the "Heart Smart" health intervention program were analyzed as a heuristic tool.…

The development of a framework for analyzing pupil talk is described and the reliability of scoring transcribed conversions using the framework discussed. Definitions and examples of the terms used in the framework are appended. (Author/YP)

Comparison using paired t-tests of parent and teacher ratings for 16 preschool children on the Receptive-Expressive Emergent Language Scale found no significant differences between parent and teacher ratings of expressive language, but a significant difference on the receptive language subscale. However, interrater reliability was relatively low…

The purpose of this study was to determine what trends exist in the identification and description of participants used in data-based studies published in "Adapted Physical Activity Quarterly" and the "Journal of Teaching in Physical Education". Data were analyzed using frequency counts for journals and time periods from the 1980s to 2005 with…

Numerous studies have shown increasing item reliabilities as an effect of the item position in personality scales. Traditionally, these context effects are analyzed based on item-total correlations. This approach neglects that trends in item reliabilities can be caused either by an increase in true score variance or by a decrease in error…

The Association of American Medical Colleges is conducting research to develop, implement, and evaluate a Medical College Admission Test (MCAT) essay testing program. Essay administration in the spring and fall of 1985 and 1986 suggested that additional research was needed on the development of topics which elicit similar skills and meet standard…

This study is an attempt at a cohesive characterization of the concept of essay reliability. As such, it takes as a basic premise that previous and current practices in reporting reliability estimates for essay tests have certain shortcomings. The study provides an analysis of these shortcomings--partly to encourage a fuller understanding of the…

The purpose of this study was to compare the relative magnitude of the variance components and generalizability coefficients derived from the Teacher Performance Assessment Instruments (TPAI) data using two different methods of data collection: (1) occasions when observers were in the classroom for simultaneous observation and (2) occasions when…