Short comparative interrupted times series (CITS) designs are increasingly being used in education research to assess the effectiveness of school-level interventions. These designs can be implemented relatively inexpensively, often drawing on publicly available data on aggregate school performance. However, the validity of this approach hinges on…

This guide is addressed to policy officials, program providers, and researchers who are seeking to: (1) identify and implement social programs backed by valid evidence of effectiveness; or (2) sponsor or conduct an evaluation to determine whether a program is effective. The guide provides a brief overview of which studies can produce valid…

As evidence-based reform becomes increasingly important in educational policy, it is becoming essential to understand how research design might contribute to reported effect sizes in experiments evaluating educational programs. The purpose of this study was to examine how methodological features such as types of publication, sample sizes, and…

This report summarizes the complex research literature on quantitative methods for assessing how impacts of educational interventions on instructional practices and student learning differ across students, educators, and schools. It also provides technical guidance about the use and interpretation of these methods. The research topics addressed…

This methodological brief is designed to provide both program operators and researchers with practical advice about how to assess a program's implementation and impact. Adapted from an article that first appeared in "The Handbook of Youth Mentoring" (DuBois and Karcher, ed. 2005), the brief focuses on the evaluation of mentoring programs, but the…

This article explores the practical relevance of pragmatism as a research paradigm through the example of a piece of pragmatic research that not only used both quantitative and qualitative research methods but also exploited the inherent duality of the data analyzed. Thus, the article aims to make the case that pragmatism as a research paradigm…

Social sciences research is entrenched with particular values, beliefs, norms, and practices that students, faculty, and researchers reproduce over time. In this article, the author argues for extending what counts as research within the social sciences to be more inclusive of differing methodologies and writing genres. Using personal narrative,…

The article proposes a sequential alternating treatments design requiring that two treatments be initially implemented in a random or counterbalanced fashion followed by a sequential change in one or both treatments across settings, subjects, or tasks. This design is appropriate when it is not feasible to obtain a traditional baseline condition.…

In order to determine the extent to which the LOGO programming language can be used as a reinforcer for general problem solving ability, this field study used a production theory approach to problem solving as a conceptual model, and then translated the model into a LOGO oriented framework. The methodology tested the use of an algorithm in a…

A Monte Carlo study was done to determine the adequate sample size for quasi-experimental regression studies, which compare regression lines for two groups and estimate their point of intersection. Populations of 1,000 subjects in each of two groups were constructed (using random normal deviates) to yield equivalent regression lines of opposite…

One goal of professional development is to improve student achievement through improved teacher practice. The middle school philosophy, which supports the rationale that student learning is inextricably interwoven into the fabric of an active learning environment, contains many promising practices that "mirror" what is often considered to be high…

Community-based educational programs for older adults are difficult to evaluate due to randomization and control factors. In order to investigate, under a quasi-experimental research design, the contribution of an educational program on retirement issues to participants' quality of life, and to study the program enrollment patterns, 21 people…

Perhaps no facet of long-term care (LTC) research has received as much attention as the evaluation of community-based LTC demonstrations. Recent thinking has called into question the validity and policy relevance of the findings of these studies, yet their efforts will continue to receive strong financial backing. The time has come for LTC…

How social studies researchers can use National Assessment of Educational Progress Public Use Data File tapes to delimit the problem of generalizability of research results is described. An example using data from a trend study of a citizenship education project is used to demonstrate the procedure. (Author/RM)

The central purpose of the present study was to evaluate several statistical techniques (analysis of variance (ANOVA) on raw gains, standardized ANOVA, standard analysis of covariance (ANCOVA), and z-score ANOVA) under various quasi-experimental conditions (three levels of reliability, three levels of sample size, three levels of gain, and the…