This paper argues for three aspects of tolerance with respect to QCA research: tolerance with respect to different approaches to QCA; producing QCA research with tolerance (work that is resistant to criticism); and for QCA researchers to be clear about the tolerance of the solutions they present -- especially in terms of calibration and truth…

Once research questions are posed, researchers must answer many a priori questions regarding research design before analysis can be performed and any conclusions can be made, including sample selection criteria, data collection method, model specification, analysis and estimation technique. The choices made by researchers along this forking path…

Conducting research with human subjects can be difficult because of limited sample sizes and small empirical effects. We demonstrate that this problem can yield patterns of results that are practically indistinguishable from flipping a coin to determine the direction of treatment effects. We use this idea of random conclusions to establish a…

Background: Stealth assessment is a learning analytics method, which leverages the collection and analysis of learners' interaction data to make real-time inferences about their learning. Employed in digital learning environments, stealth assessment helps researchers, educators, and teachers evaluate learners' competencies and customize the…

The methodology of single-case experimental designs (SCED) has been expanding its efforts toward rigorous design tactics to address a variety of research questions related to intervention effectiveness. Effect size indicators appropriate to quantify the magnitude and the direction of interventions have been recommended and intensively studied for…

Clinical trials have different designs: In late stage drug development, the parallel trial design is the most frequent one; however, the crossover design is not rare; different techniques are used to analyse their results. Although both designs measure the same treatment effect, combining parallel and crossover trials in a meta-analysis is not…

In this commentary, we briefly review the collective effort of design researchers to weave theory with empirical results, in order to gain a better understanding of the processes of learning. We seek to respond to this challenging agenda by centring on the evolution of one sub-field: namely that which involves investigations within a…

Taped problems is an intervention strategy for addressing mathematics fluency that has been evaluated in multiple single-case design studies. Although its efficacy has been supported in individual studies, no comprehensive quantitative synthesis has been conducted on taped problems. The purpose of this study was to synthesize the literature that…

The current study aimed to review studies on computational thinking (CT) indexed in Web of Science (WOS) and ERIC databases. A thorough search in electronic databases revealed 96 studies on computational thinking which were published between 2006 and 2016. Studies were exposed to a quantitative content analysis through using an article control…

This methodological brief introduces basic procedures and issues for conducting a high-quality meta-analysis in gifted education. Specifically, we discuss issues such as how to select a topic and formulate research problems, search for and identify qualified studies, code studies and extract data, choose and calculate effect sizes, analyze data,…

Every year the U.S. Department of Education reports for each state in the country the grade levels, subject areas, and geographic areas that have experienced teacher shortages (U.S. Department of Education, Office of Postsecondary Education, 2015). A teacher shortage occurs when the number of teachers available in a specific grade, subject matter…

It is common in regression discontinuity analysis to control for high order (third, fourth, or higher) polynomials of the forcing variable. We argue that estimators for causal effects based on such methods can be misleading, and we recommend researchers do not use them, and instead use estimators based on local linear or quadratic polynomials or…

A valuable extension of the single-rating regression discontinuity design (RDD) is a multiple-rating RDD (MRRDD). To date, four main methods have been used to estimate average treatment effects at the multiple treatment frontiers of an MRRDD: the "surface" method, the "frontier" method, the "binding-score" method, and…

In this article, we test an alternative approach to creating a counterfactual basis for estimating individual and average treatment effects. Instead of using control/comparison groups or before-measures, the so-called Counterfactual as Self-Estimated by Program Participants (CSEPP) relies on program participants' self-estimations of their own…

The call for researchers to report and interpret effect sizes and their corresponding confidence intervals has never been stronger. However, there is confusion in the literature on the definition of effect size, and consequently the term is used inconsistently. We propose a definition for effect size, discuss 3 facets of effect size (dimension,…