The authors investigated the performance of 172 Grade 4 students (9 to 10 years) over 12 months on a 36-item test that comprised items from 6 distinct graphical languages (e.g., maps) commonly used to convey mathematical information. Results revealed (a) difficulties in Grade 4 students' capacity to decode a variety of graphics, (b) significant…

We obtained data from pre- and postcourse questionnaires given in an Introduction to the Psychology Major course taught for 10 semesters and compared these data with institutional outcomes concerning the students' last known major and their graduation status. We found the questionnaire reliably measured (a) vocational identity, (b) knowledge of…

This study evaluates the reliability of self-assessment as a measure of computer competence. This evaluation is carried out in response to recent research which has employed self-reported ratings as the sole indicator of students' computer competence. To evaluate the reliability of self-assessed computer competence, the scores achieved by students…

This study examined the impact of rater agreement on decisions concerning the alignment between the "Golden State Examination in High School Mathematics" (California Department of Education, 2001a) and the University of California (UC) "Statement on Competencies in Mathematics Expected of Entering College Students" (Academic…

Computer-generated anthropomorphic characters are a growing type of communicator that is deployed in digital communication environments. An essential theoretical question is how people identify humanlike but clearly artificial, hence humanoid, entities in comparison to natural human ones. This identity categorization inquiry was approached under…

The Childhood Asperger Syndrome Test (CAST) is a 37-item parental self-completion questionnaire designed to screen for high-functioning autism spectrum conditions in epidemiological research. The CAST has previously demonstrated good accuracy for use as a screening test, with high sensitivity in studies with primary school aged children in…

A measurement approach called generalizability theory (G-theory) is an important alternative to the more familiar classical measurement theory that yields less useful coefficients such as alpha or the KR-20 coefficient. G-theory is a theory about the dependability of behavioral measurements that allows the simultaneous estimation of multiple…

This paper outlines the development of a performance assessment measure of language speaking ability, the Language Ability Assessment System (LAAS), which is highly reliable and can be examined for reliability through modern measurement theories, such as generalizability theory (G-theory) and the many-facet Rasch theory. LAAS was developed to…

A study of the reliability of the proficiency ratings scale and techniques used by three federal government agencies--the Central Intelligence Agency, the Defense Language Institute, and the Foreign Service Institute (FSI)--to test employees' oral language proficiency in French and German had two randomly selected two-person teams of testers from…

The development and validation of a Portuguese oral language test are described. The test consisted of five item types: personal conversation, giving directions, description of picture sequences, topical discourse, and oral task completion based on printed instructions. Three preliminary forms of the test were administered to a group of language…

The purpose of this paper is to discuss ways in which developmental psychology suffers from the lack of an appropriate technology of measurement and statistical analysis. The paper begins by noting that developmental psychology is the study of change; that individuals develop through a succession of "stages" which are separated by…

Guidelines for the use of observation instruments in pre-and in-service teachers of the handicapped are provided. It is explained that guidelines have been developed according to the following five principles: (1) the observation instrument must be relevant (including empirical, normative, and interpretative relevance); (2) methods of gathering…

The Answer-Until-Correct (AUC) procedure has been proposed in order to increase the reliability of multiple-choice items. A model for examinees' behavior when they must respond to each item until they answer it correctly is presented. An expression for the reliability of AUC items, as a function of the characteristics of the item and the scoring…

Differences in procedures used by academic institutions and government agencies in administering the American Council on the Teaching of Foreign Languages' Oral Proficiency Interview test are examined, and results and implications of two studies of interrater reliability are discussed. (MSE)

Examined the generalizability of written-response scores on the English 30 diploma examination administered to Alberta 12th-grade students. Student scores differed as a function of rater, but this variance component was small across two tasks and two administrations; score generalizability was high using a two-rater system; and scale variability…