The Programme for International Student Assessment (PISA) assesses the knowledge and skills in mathematics, reading and science of 15-year-old pupils in countries around the world. PISA is run by the Organisation for Economic Co-operation and Development (OECD), and assessment is typically undertaken every 3 years, allowing us to chart how…

This rejoinder keeps the original focus on statistical computing pertaining to the correlation of student achievement between mathematics and science from the Trend in Mathematics and Science Study (TIMSS). Albeit the availability of student performance data in TIMSS and the emphasis of the inter-subject connection in the Next Generation Science…

There are numerous statistical procedures for detecting items that function differently across subgroups of examinees that take a test or survey. However, in endeavouring to detect items that may function differentially, selection of the statistical method is only one of many important decisions. In this article, we discuss the important decisions…

In this paper, the authors examine some of the ways that different types of non-equivalent comparison groups can be used to strengthen causal inferences based on regression discontinuity design (RDD). First, they consider a design that incorporates pre-test data on assignment scores and outcomes that were collected either before the treatment…

In recent years, the regression discontinuity design (RDD) has gained widespread recognition as a quasi-experimental method that when used correctly, can produce internally valid estimates of causal effects of a treatment, a program or an intervention (hereafter referred to as treatment effects). In an RDD study, subjects or groups of subjects…

Educational research focused on questioning techniques used in classroom settings is quite extensive. However, research regarding a teacher's ability to generate research questions is virtually nonexistent. Posing research or "testable" questions is a key component of inquiry-based instruction, and teachers must be able to both generate…

Use of multivariate analysis (e.g., multivariate analysis of variance) is common when normally distributed outcomes are collected in intervention research. However, when mixed responses--a set of normal and binary outcomes--are collected, standard multivariate analyses are no longer suitable. While mixed responses are often obtained in…

In this paper, we examine the validity and precision of two nonexperimental study designs (NXDs) that can be used in educational evaluation: the comparative interrupted time series (CITS) design and the difference-in-difference (DD) design. In a CITS design, program impacts are evaluated by looking at whether the treatment group deviates from its…

The Early Childhood Longitudinal Study, Kindergarten Class of 2010-11 (ECLS-K:2011), is collecting information about the early educational experiences of a nationally representative sample of children who were in kindergarten or who were of kindergarten age in ungraded classrooms or schools in the 2010-11 school year. The data collection began in…

Classroom context and climate are inherently classroom-level (L2) constructs, but applied researchers sometimes--inappropriately--represent them by student-level (L1) responses in single-level models rather than more appropriate multilevel models. Here we focus on important conceptual issues (distinctions between climate and contextual variables;…

This paper examines the impact of a gifted program on retention in an urban school district using a regression discontinuity design. Gifted programs often employ IQ thresholds for admission, with those above the threshold being admitted. One common problem with the RD design arises if the forcing variable (the IQ score) is manipulated, thus…

Pascarella (J Coll Stud Dev 47:508-520, 2006) has called for an increase in use of longitudinal data with pretest-posttest design when studying effects on college students. However, such designs that use multiple measures to document change are vulnerable to an important threat to internal validity, regression to the mean. Herein, we discuss a…

A Monte Carlo study compared the statistical performance of standard and robust multilevel mediation analysis methods to test indirect effects for a cluster randomized experimental design under various departures from normality. The performance of these methods was examined for an upper-level mediation process, where the indirect effect is a fixed…

This study investigated the comprehension and effectiveness of teaching formal, probabilistic decision-making skills to middle school students. Two specific objectives were to determine (1) if middle school students can comprehend a probabilistic decision-making approach, and (2) if exposure to the modeling approaches improves middle school…

This article examines theoretical and empirical issues related to the statistical power of impact estimates under clustered regression discontinuity (RD) designs. The theory is grounded in the causal inference and hierarchical linear modeling literature, and the empirical work focuses on common designs used in education research to test…