Lottery-based identification strategies offer potential for generating the next generation of evidence on U.S. early education programs. The authors' collaborative network of five research teams applying this design in early education settings and methods experts has identified six challenges that need to be carefully considered in this next…

Nationally, teachers have experienced major changes in how their performance is evaluated. Teacher satisfaction with the evaluation system likely influences whether they will use the evaluation data to make instructional improvements, yet we know little about teachers' perceptions of their evaluations. Using survey data from Chicago Public…

Given the rapid adoption of machine learning methods by education researchers, and the growing acknowledgment of their inherent risks, there is an urgent need for tailored methodological guidance on how to improve and evaluate the validity of inferences drawn from these methods. Drawing on an integrative literature review and extending a…

Several state policies link high-stakes consequences to teacher evaluations, which tend to be heavily weighted by observation scores. However, research has only recently investigated the validity of these scores in field settings. This study examines the sensitivity of teacher observation scores to the number of observations assigned by state…

We investigated how the transition to remote instruction during the COVID-19 pandemic affected students' engagement, self-appraisals, and learning in advanced placement (AP) Statistics courses. Participants included 681 (M[subscript age]=16.7 years, SD[subscript age]=0.90; %female=55.4) students enrolled in the course during 2017-2018 (N=266),…

Higher education in India was caught completely unawares by the COVID-19 pandemic and the necessitated closure of educational institutions. Despite almost a decade of experience with online and distance learning at some top-tier and private institutions, the vast majority were unprepared and looked for quick solutions for different components of…

We investigated how evaluation standards employed in performance tests affect the impact of performance goals (here focused on appearance) on academic cheating. Thereby, we assumed that appearance goals would lead to increased cheating only if students' performance was presumably evaluated based on results rather than on the strategies they…

Although experimental evaluations have been labeled the "gold standard" of evidence for policy (U.S. Department of Education, 2003), evaluations without an analysis of costs are not sufficient for policymaking (Monk, 1995; Ross et al., 2007). Funding organizations now require cost-effectiveness data in most evaluations of effects. Yet,…

This manuscript describes our effort to apply a novel approach to understanding student outcomes associated with a schoolwide antiracist intervention. We report a multimethod quantitative approach to evaluate a 10-week antiracist intervention designed and implemented by school staff by examining patterns of student intervention engagement and…

We draw on administrative data from the country of Colombia to assess differences in student learning in online and traditional on-campus college programs. The Colombian context is uniquely suited to study this topic, as students take a compulsory exit examination at the end of their studies. We can therefore directly compare the performance on…

Student evaluations of teaching are widely used to assess instructors and courses. Using a model-based approach and Bayesian methods, we examine how the direction of the scale, labels on scales, and the number of options affect the ratings. We conduct a within-participants experiment in which respondents evaluate instructors and lectures using…

Registered reports engender a review process in which reviewers evaluate the merits of the research question rather than the magnitude or direction of the results. In this process, authors first submit introduction, methods, and prespecified analyses sections of an article. Next, an initial peer-review process assesses the merits of the research…

Prior work has shown that students' self-reported levels on social-emotional measures predict achievement levels and gains, but we have little evidence on whether within-student changes in self-reports on social-emotional surveys are predictive of changes in theoretically related academic and behavioral outcomes. We use large-scale data from the…

The Gifted and Talented program in New York City is one of the largest and longest running programs for gifted students in the nation. Yet little is known about its effects on student outcomes. Using student-level administrative data of New York City public school students between the 2010--2011 and 2018-2019 academic years, we studied the effects…

Diversity, equity, and inclusion (DEI) issues are urgent in education. We developed and evaluated a massive open online course (N = 963) with embedded equity simulations that attempted to equip educators with equity teaching practices. Applying a structural topic model (STM)--a type of natural language processing (NLP)--we examined how…