This report shows that there are many ways to confirm the accuracy, reliability, and validity of student self-reports. Examples from higher education and from public opinion polls and general surveys demonstrate some of the common sources of measurement errors and errors of substance. Part 1 of the report summarizes a few highlights from the…

Uses of discipline and program cost data by Indiana private colleges and universities are discussed, and cost data are presented. Attention is directed to the immediate and longer range uses of the data, the applicability of the methods to smaller institutions, reliability of the data, and the future of this analysis. The findings were used for…

This study attempts to determine whether there exists a negative interconnection between the creative and testable nature-of-science (NOS) conceptions in college students' conceptual ecology by investigating, through a pair of IHV-assisted teaching experiments, the effect of raising the status of each NOS conception in students' conceptual ecology…

U.S. universities and colleges offering distance education courses have increased immensely since 1998, and by 2004 it was expected that distance learners will constitute about 14% of all those enrolled in degree programs. In its preliminary review of distance learning, the Institute for Higher Education Policy (1998) emphasized the need for…

Observer reliability and the confusability of codes, two sources of error in the collection of classroom observational data, are examined. Confusability is defined as the extent to which one code is mistakenly recorded as another code. Observational data were collected in each of 172 first grade and 171 third grade Follow Through and comparison…

A study was carried out to ascertain whether language instructors or naive judges were more reliable in judging oral proficiency. Fifteen students were chosen from the ESL (English as a second language) center at Southern Illinois University to record a tape while reading passages in English. The tape and a questionnaire were administered to 70…

The object of this research was to establish a reliable test battery to determine probable future success for those persons who wish to be trained as expert classroom observers. The focus of this investigation was to predict coding performance. This report presents the findings of a replication of an earlier study that indicated that coder…

To improve the reliability of selection interviews, the faculty of the University of Texas Medical Branch physician's assistant program developed a structured fourteen-category interview. The thirty-minute interview was used to select from 94 applicants; each applicant was interviewed three times and independently rated on a five-point scale of…

This training text is designed to introduce program evaluation concepts and terminology to beginning program evaluators for state-federal vocational rehabilitation programs. Taking a research approach, the text covers the principles underlying program evaluation, and is geared to give the reader a basic understanding of how these principles can be…

This is the first of four volumes reporting phase 2 of a two-phase project to examine the competency-based vocational teacher education movement and analyze the current state of the art (phase 1), and to assess the implications for preservice, inservice, and leadership level professional development programs in career and occupational education in…

Patterns of stability of behaviors in classrooms with varying types of structure were investigated by means of a low-inference rating scale applicable to both traditional and nontraditional classrooms. Utilizing a research design which controls for subject matter, period of day, and day of week, eighteen observations were made in each of six…

After an interview procedure used with educable mentally retarded children is described, the findings of an examination of some collected interview data are reported in this study of the Computer Based Project for the Evaluation of Media for the Handicapped. An ex post facto analysis revealed a moderately high correlation between interviewer…

The first task of the study was to develop valid, short-cut methods of indexing the narrative content of evaluation reports that would extract the differentiating information contained in evaluative comments simply, reliably, and with as good classification accuracy as the longer initial procedure. In the second task, the original inter-indexer…

The purpose of this City Data study was to explore sources of data outside the National Evaluation study of Follow Through to determine the usefulness of such information in the overall assessment. The two major objectives of the study were to judge the representativeness of the Project Follow Through sample and to check on the credibility of…

Discusses current issues in measuring library effectiveness, including rapid technological changes and increasing obsolescence in librarians' competencies; increased pressure for accountability; tools to help libraries become what they determine they should be rather than conforming to national standards; and how to determine the validity and…