Multisite field experiments using the (generalized) randomized block design that assign treatments to individuals within sites are common in education and the social sciences. Under this design, there are two possible estimands of interest and they differ based on whether sites or blocks have fixed or random effects. When the average treatment…

Quantitative Critical Race Theory (QuantCrit) is a burgeoning field of study seeking to challenge and improve the use of statistical data in social research. It pulls lessons and insights from Critical Race Theory and applies them to understanding social challenges. In this paper, we aim to improve the quality of quantitative research produced by…

Cluster randomized experiments are ubiquitous in modern education research. Although a variety of modeling approaches are used to analyze these data, perhaps the most common methodology is a normal mixed effects model where some effects, such as the treatment effect, are regarded as fixed, and others, such as the effect of group random assignment…

Experimental evaluations that involve the educational system usually involve a hierarchical structure (students are nested within classrooms that are nested within schools, etc.). Concerns about contamination, where research subjects receive certain features of an intervention intended for subjects in a different experimental group, have often led…

When evaluators plan site-randomized experiments, they must conduct the appropriate statistical power analyses. These analyses are most likely to be valid when they are based on data from the jurisdictions in which the studies are to be conducted. In this method note, we provide software code, in the form of a SAS macro, for producing statistical…

The paper shows how the methodology for a pedagogical experiment can be improved through including the pre-research stage. If the experiment has the form of a test procedure, an improvement of methodology can be achieved using for example the methods of statistical and didactic analysis of tests which are traditionally used in other areas, i.e.…

As evidence-based reform becomes increasingly important in educational policy, it is becoming essential to understand how research design might contribute to reported effect sizes in experiments evaluating educational programs. The purpose of this study was to examine how methodological features such as types of publication, sample sizes, and…

Article discussed the relevance for educational research found in the Type 11 error, one identified as a correct rejection of a 2-tailed test of hypothesis followed by an incorrect directional decision. (Author/RK)

This paper presents the results of an experiment conducted to assess the affects of teaching recursion in two disjoint, non-consecutive units of instruction. One group of students was taught basic and advanced recursion topics in four consecutive class periods, while a second group was taught recursion in two two-period blocks that were separated…

This paper presents a number of inferential statistical models that may aid the operant investigator in presenting his data. (Author/RK)

The problem of spurious significance in multivariate exploratory research is discussed. When a very large number of statistical tests are performed, many tests will be significant on the basis of chance alone. To counter this problem, the use of two sign tests to analyze sets of results has been suggested; the chance expectation [CE test] assesses…

Exploratory research with large numbers of variables and even larger numbers of relationships is likely to result in frequent Type II errors; that is falsely accepting an incorrect null hypotheses. There are three ways of reducing Type II errors: (1) choosing a lower significance level, which then increases the probability of Type I error; (2)…

The purpose of an 8-week summer training institute was to provide 25 competent professional school personnel the necessary methodological skills to initiate, conduct, and report experiments on school learning and instruction. All trainees held administrative posts with responsibilities for curriculum research and development and/or instruction. A…

Reviews empirical education research trends in West Germany since 1965. Reports on the movement from normative research to empirical research in educational settings. Considers research priorities and the primacy of the University of Constance (West Germany) in developments focusing on analyzing the factual order of the educational system rather…