Purpose: Course evaluations are formative and are used to evaluate learnings of the students for a course. Anomalies in the evaluation process can lead to a faulty educational outcome. Learning analytics and educational data mining provide a set of techniques that can be conveniently applied to extensive data collected as part of the evaluation…

RobotReviewer is a tool for automatically assessing the risk of bias in randomized controlled trials, but there is limited evidence of its reliability. We evaluated the agreement between RobotReviewer and humans regarding the risk of bias assessment based on 1955 randomized controlled trials. The risk of bias in these trials was assessed via two…

This colloquium brings information about a second cycle of OECD's International Early Learning and Well-being Study (IELS) to the early childhood community, and offers a further critique of the approach to comparative research that the IELS embodies.

Although parallel analysis has been found to be an accurate method for determining the number of factors in many conditions with complete data, its application under missing data is limited. The existing literature recommends that, after using an appropriate multiple imputation method, researchers either apply parallel analysis to every imputed…

Speculative design, as a diverse set of methods that aim to offer critique, can be challenging to engage productively. In this design case, we share how a prior, stalled design project--an ambitious vision of interdisciplinary design education partnered with business and housing development projects in Santa Fe, New Mexico--provided compelling…

To support and better understand how to scale effectively, in 2020, the Millions Learning project at the Center for Universal Education (CUE) at Brookings joined the Global Partnership for Education's (GPE) Knowledge and Innovation Exchange (KIX), a joint partnership between GPE and the International Development Research Centre (IDRC), to…

When conducting a postal survey, a traditional recommendation is to use paper postage stamps instead of an automated postage system, to make sure that invitations have a more personal and attractive appearance. In this research note, we investigate whether this traditional recommendation is still valid. In the autumn of 2020, a postal survey was…

Audit correspondence studies are field experiments that test for discriminatory behavior in active markets. Researchers measure discrimination by comparing how responsive individuals ("audited units") are to correspondences from different types of people. This article elaborates on the tradeoffs researchers face between sending audited…

The growing interest in video research and new technologies for recording human interaction has stirred debates about intrusiveness and 'reactivity' understood as researcher-derived changes in subjects. In addition to a plethora of concepts referring to such effects in extant literature, different ontological and epistemological positions provide…

Educational research is reflective of the nature and structure of national and regional education systems and their historical evolution. Educational technology research, as an area within educational research, reflects this case particularly prominently. Although individual countries and regions have varying research traditions, the publication…

This article examines an important and thorny problem in interview research: How to assess whether what people say motivated their actions actually did so? We ask three questions: What specific challenges are at play? How have researchers addressed them? And how should those strategies be evaluated? We argue that such research faces at least five…

This article seeks to move beyond the Euro/North-centrism recurrent in methodological discussions on what we may learn from the COVID-19 pandemic. Such debates often centre on uncertainty and involuntary immobility -- aspects which are hardly new for many researchers. In this article, we argue that the pandemic offers an opportunity to rethink…

Outside of education, many academic disciplines developed and promulgated checklists to evaluate the quality of qualitative studies. This article used an embedded mixed methods approach in three stages: development of a conceptual framework reviewing 62 guidelines; a content analysis of key areas of the guidelines; and review of dissertations to…

In this study, it was aimed to examine the effect of missing data in different patterns and sizes on test equating methods under the NEAT design for different factors. For this purpose, as part of this study, factors such as sample size, average difficulty level difference between the test forms, difference between the ability distribution,…

Over the last 30 years, there has been substantial debate about the practical, ethical and epistemological issues uniquely associated with qualitative data sharing. In this paper, we contribute to these debates by examining established data sharing practices in Conversation Analysis (CA). CA is an approach to the analysis of social interaction…