In the complex world of educational policy, causal inferences will be debated. As we review non-experimental designs in educational policy, we focus on how to clarify and focus the terms of debate. We begin by presenting the potential outcomes/counterfactual framework and then describe approximations to the counterfactual generated from the…

The problem with most commissioned studies is that they are generally anchored on research protocols, guidelines and theoretical underpinnings imposed by research grant makers or the external Principal Investigator's (PI) own research design preferences. Furthermore, the problem is that researchers who are recruited during various phases of…

Qualitative research has been strongly affected by the COVID-19 pandemic, highlighting the possibilities that Voice over Internet Protocol (VoIP) technologies such as Skype, WhatsApp, and Zoom offer to qualitative scholars. Based on the experience of using such technologies to collect qualitative data for our PhD studies, we present how we dealt…

Many processes and practices in the field of health professions education have been based more on tradition and assumption than on evidence and theory. As the field matures, researchers are increasingly seeking evidence to support various teaching and assessment methods. However, there is a tendency to focus on a limited set of topics, leaving…

Meta-analytic reviews collect available empirical studies on a specified domain and calculate the average effect of a factor. Educators as well as researchers exploring a new domain of inquiry may rely on the conclusions from meta-analytic reviews rather than reading multiple primary studies. This article calls for caution in this regard because…

Missing data incidents are common in experimental studies of motor learning and development. Inadequate handling of missing data may lead to serious problems, such as addition of bias, reduction in power, and so on. Thus, this study aimed to conduct a systematic review of the past (2007) and present (2017) practices used for reporting and…

In this article, we examine whether cross-national studies disclose enough information for independent researchers to evaluate the validity and reliability of the findings (evaluation transparency) or to perform a direct replication (replicability transparency). The first contribution is theoretical. We develop a heuristic theoretical model…

Publication pressure has been touted to promote questionable research practices (QRP) and scientific or research misconduct (RM). However, logically attractively as it is, there is no unequivocal evidence for this notion, and empirical studies have produced conflicting results. Other than difficulties in obtaining unbiased empirical data, a direct…

Purpose: Various aspects of speech production related to auditory-motor integration and learning have been examined through auditory feedback perturbation paradigms in which participants' acoustic speech output is experimentally altered and played back via earphones/headphones "in real time." Scientific rigor requires high precision in…

In this paper, we explore themes that emerged while reading "How does one recognize contextual mitigating factors (CMFs) as a basis to understand and arrive at better approaches to research designs" (Gallard Martínez et al. this issue). Specifically, we consider the potential for the CMF analysis method to help researchers be more…

Recurrent pain and school failures are common problems in children visiting the school nurses office. The overall aim of the current study was to investigate the relationship between recurrent pain and academic achievement in school-aged children. Literature was searched in seven electronic databases and in relevant bibliographies. Study…

Realist evaluation (RE) is a research design increasingly used in program evaluation, that aims to explore and understand the influence of context and underlying mechanisms on intervention or program outcomes. Several methodological challenges, however, are associated with this approach. This article summarizes RE key principles and examines some…

Education policy makers must make decisions under uncertainty. Thus, how they think about risks has important implications for resource allocation, interventions, innovation, and the information that is provided to the public. In this policy brief we illustrate how the standard of evidence for making decisions can be quite inconsistently applied,…

Qualitative research (QR) has gained visibility and acceptance in the field of special education due to early efforts to identify quality indicators focused on technical and methodological aspects of QR. Whereas these indicators focused on credibility and trustworthiness of data, this article articulates additional QR quality indicators to enhance…

Social research methodologists have postulated that the transparency of survey procedures and data processing is mandatory for assessing the Total Survey Error. Recent analyses of data from cross-national surveys have demonstrated an increase in the quality of documentation reports over time and significant differences in documentation quality…