Even when raters demonstrate agreement in the use of a measure, limited score variability or violation of often-ignored statistical assumptions can result in lower reliability estimates than intuitively expected. This article uses data drawn from two randomized controlled trials of schema therapy and cognitive behavioral therapy for the treatment…

Jeffrey Beall's blog listing of potential predatory journals and publishers, as well as his "Criteria for Determining Predatory Open-Access (OA) Publishers" are often looked at as tools to help researchers avoid publishing in predatory journals. While these "Criteria" have brought a greater awareness of OA predatory journals,…

This guide is designed to help researchers make sure that their research reports include enough information about study measures so that readers can assess the quality of the study's methods and results. The guide also provides examples of write-ups about measures and suggests resources for learning more about these topics. The guide assumes…

This article describes the core features of outcome research and then explores issues confronting researchers who engage in outcome studies. Using an intervention research perspective, descriptive and explanatory methods are distinguished. Emphasis is placed on the counterfactual causal perspective, designing programs that fit culture and context,…

The reliability of a new scale to assess Autistic Disorder, Pervasive Developmental Disorder, Not Otherwise Specified (PDD-NOS), and Asperger's Disorder in children was examined. Parents or other caregivers rated symptoms of 207 children between 2 and 16 years of age. The scale, which had 40 items in the final version, correlated highly with…

Quantitative behaviour analysis requires the classification of behaviour to produce the basic data. This can be challenging when the theoretical taxonomy does not match observational limitations, or if a theoretical taxonomy is unavailable. Binary keys allow qualitative observation to be used to modify a theoretical taxonomy to produce a practical…

This paper contends that behavior observation data relating to speech-language pathology are reliable if they are not affected by differences among observers or other variations in the recording context. The theoretical bases of methods used to estimate reliability for observational data are reviewed, and suggestions are provided for improving the…

Describes pilot studies of the Edinburgh Extended Glasgow Outcome Scale (EEGOS), designed to retain the advantages of the GOS (a measure commonly used in head injury research) but to allow comparison of recovery patterns in behavioral, cognitive, and physical function. Studies show that the interrater reliability of the EEGOS is comparable to that…

In this study, self-reports of 34 individuals with developmental disabilities (DD) were compared with proxy ratings from family and providers. Correlations between the ratings of individuals with DD and the proxy raters were low, as were the correlations between family members and providers. In all scales except "cognition," the individual with DD…

The study, with 79 rehabilitation therapists evaluating 21 single-subject graphs, found that the low interrater agreement often associated with visual analysis of single-subject data may be improved by simple supplements (such as trend lines) to visually inspected charts. (Author/DB)

Based upon the assumption that the process of peer review of publications and research is flawed, interrater reliability of reviews of 188 research proposals submitted for funding at a major university was studied. The eight dimensions rated were: (1) significance of the research; (2) clarity and reasonableness of the objectives; (3)…

Examines the tendency of observers to make less reliable recordings of behavorial events when a calibrating observer is absent. Using four different multicategory systems, 26 experienced observers coded 200 hours of videotaped family interactions. Concludes that observers lapse into a less attentive "set" prior to coding without a…

Four experienced stuttering researchers viewed videodisks of spontaneous speech from chronic stutterers and attempted to locate the precise onset and offset of individual stuttering events. Results showed interjudge disagreements that challenge the reliability and validity of onset and offset judgments. Highly agreed stuttering events were…

The Correct Information Unity analysis for measuring the communicative information and efficiency of connected speech was applied to the naturally occurring conversation of a person with moderate aphasia. Results indicated low intrarater and interrater reliability although reliability of word counts was good. Most rater disagreements resulted from…

This study was designed as a reconsideration of the weights used in evaluative decisions made with regard to research proposals submitted for funding at a major state university. The specific objective of the study was to determine whether the actual weights for components used in the evaluation of the proposals differed from a priori weights…