Little is known empirically about intraclass correlations (ICCs) for multisite cluster randomized trial (MSCRT) designs, particularly in science education. In this study, ICCs suitable for science achievement studies using a three-level (students in schools in districts) MSCRT design that block on district are estimated and examined. Estimates of…

Scientifically based research used to inform evidence based school reform efforts has been required by the federal government in order to receive grant funding since the reenactment of No Child Left Behind (2002). Educational evaluators are thus faced with the challenge to use rigorous research designs to establish causal relationships. However,…

In the late 1990s, teachers in five pilot districts in Ohio were trained during the Baldrige in Education Initiative (BiE IN). Training included Baldrige's theory, quality process and quality tools. The study was a follow-up to determine the effect of the use of the Ohio Baldrige Initiative training in the pilot districts (and other early…

The equipercentile assumption states that students in traditional classrooms who receive no other instructional assistance, will maintain their relative rank order over time. To test this assumption, fall to fall test results on the SRA Achievement Tests were obtained for grades 2-3, and 6-7. Total reading and total mathematics growth scale values…

As a new program is developed, information must be collected to identify weaknesses, to guide the staff in the revision process, and to formulate decision-making procedures, i.e., formative evaluation techniques are essential. A set of criteria for the information needed in formative evaluation is set-up and a practical strategy for meeting these…

Discusses the need for awareness of real and potential problems in educational evaluation approaches, kinds of problems there are with true experimental designs and with quasi-experimental designs, and the criteria applied by the Joint Dissemination Review Panel to its review of educational programs. (MF)

Seven considerations in designing the Title I Evaluation and Reporting System are reviewed: satisfying state, local, and federal needs; summative evaluation; differences in Title I subprograms; local use of different achievement tests; comparative analysis data; use of approved evaluation designs; and use of a common reporting metric. (MH)

Twelve common errors which can invalidate an otherwise sound evaluation are identified, and ways to avoid them are presented. The hazards are: (1) grade-equivalent scores; (2) inappropriate statistical adjustments with nonequivalent control groups; (3) administering norm-referenced tests at inappropriate times of the school year; (4) inappropriate…

This study explored the assumption that the percentile rank of a group of students should be expected to remain the same on a post-test as on a pretest when the treatment of the group in the intervening period was the same as the treatment of the norms population. This assumption is basic to the use of the norm-referenced model for the assessment…

A new Joint Comprehensive Evaluation System for the assessment of 15 different federal programs has been developed by the U.S. Office of Education. In this system these diverse program services will be thought of as resources available to meet the needs of critical target groups. Using this approach, a set of nine crucial questions that need to be…

The Elementary and Secondary Education Act Title I Evaluation and Reporting System is a method for giving a federally funded project in reading or math an overall score on its cognitive effectiveness. This System introduced the Normal Curve Equivalent (NCE) as an aid in aggregating Title I program scores across states and nationwide regardless of…

This paper outlines the design of two Comparative Studies of Phase IV of the Individually Guided Education (IGE) Evaluation Project. More than 2,000 elementary schools in 25 states use the IGE system. The Evaluation Project was designed to gain a comprehensive view of the system's operation and effectiveness. Phase IV investigated pupil outcomes,…

Ninth grade Elementary Secondary Education Act (ESEA) Title I evaluation results in reading and English-writing projects were compared, using the RMC Model A with local empirical norms, Model A with national interpolated norms, and Model B. The Comprehensive Tests of Basic Skills (CTBS)--Total Reading and Total Language subtests--were administered…

The Bilingual Education Evaluation System was developed to help local bilingual education projects overcome evaluation obstacles and design sound, useful evaluations within the constraints of available funding and with responsiveness to current federal legislation and regulations. The evaluation system proposed involves a process component, an…