Although evaluators often use an interrupted time series (ITS) design to test hypotheses about program effects, there are few empirical tests of the design's validity. We take a randomized experiment on an educational topic and compare its effects to those from a comparative ITS (CITS) design that uses the same treatment group as the experiment…

The goal of this research is to make sensitivity analysis accessible not only to empirical researchers but also to the various stakeholders for whom educational evaluations are conducted. To do this it derives anchors for the omitted variable (OV)-program participation association intrinsically, using the Love plot to present a wide range of…

The use of growth-modeling analysis (GMA)--including hierarchical linear models, latent growth models, and general estimating equations--to evaluate interventions in psychology, psychiatry, and prevention science has grown rapidly over the last decade. However, an effect size associated with the difference between the trajectories of the…

We use unique administrative data from a leading Italian university to estimate whether the use of admission tests and conditional progression schemes are effective strategies to select high-performing students. Previous work, which has been predominantly correlational, has focused on the effect of selectivity policies on widening university…

Differential Item Functioning (DIF) is a group-related item/test characteristic targeting the issue of item/test bias. In this dissertation, a new DIF index is described that has three parts: (1) a DIF measure indicating the degree of different (item) performances between the reference and focal groups, (2) a DIF direction statement about the DIF…

The 4Rs Program (Reading, Writing, Respect and Resolution) is a "dual focus" whole school universal intervention designed to promote literacy development and social-emotional learning, that is currently being rigorously evaluated using a school-randomized trial of 18 elementary schools (9 intervention, 9 control) in New York City. The…

Among the different types of experimental design are two general categories: true experimental designs and quasi- experimental designs. True experimental designs include more than one purposively created group, common measured outcomes, and random assignment. Quasi-experimental designs are commonly used when random assignment is not practical or…

This study assesses the extent of residential segregation among 15,246 people diagnosed with psychiatric disabilities and receiving Medicaid (MA) in Philadelphia, and an identically sized group of MA recipients serving as matched controls. Results indicate that overall levels of residential segregation among this group were modest at their most…

Matching, randomization, random sampling, and other methods of controlling extraneous variation are discussed. The purpose was to specify the benefits of randomization in estimating causal effects of treatments. It is concluded that randomization should be employed whenever possible, but the use of carefully controlled nonrandomized data to…

The estimation and evaluation of the effects of alternative educational or social policies is a major purpose of decision oriented, comparative evaluation studies. This paper reviews the rationale underlying the definition and interpretation of various measures of effect magnitude and examines their relevance to evalution research. (Author)

The Evaluation of the Emergency School Assistance Program (ESAP) for the 1971-72 school year is the first application of full-blown experimental design with randomized experimental and control cases in a federal evaluation of a large scale program. It is also one of the very few evaluations which has shown that federal programs can raise tested…

The statistical technique, "Zero-One Linear Programming," that has successfully been used to create multiple tests with similar characteristics (e.g., item difficulties, test information and test specifications) in the area of educational measurement, was deemed to be a suitable method for creating multiple sets of matched samples to be…

THIS REVIEW OF THE PROBLEMS IN DESIGN AND INTERPRETATION OF RESEARCH ON HUMAN RELATIONS TRAINING, PARTICULARLY THAT USING T GROUPS AND SENSITIVITY TRAINING, REFERS TO STUDIES IN THE FIELD. PROBLEMS CONSIDERED ARE--THE PROBLEM OF CONTROLS (VOLUNTEER PARTICIPANTS, PREDICTION OF OUTCOMES), TEMPORAL CHANGE IN TRAINING OUTCOMES (RELATIONSHIP OF CHANGE…