Careless responding is a bias in survey responses that disregards the actual item content, constituting a threat to the factor structure, reliability, and validity of psychological measurements. Different approaches have been proposed to detect aberrant responses such as probing questions that directly assess test-taking behavior (e.g., bogus…

Common method variance refers to the amount of uncontrolled systematic error leading to biased estimates of scale reliability and validity and to spurious covariance shared among variables due to common method and/or common source employed in survey-based researches. As the extended use of self-report questionnaires is inevitable, numerous studies…

Two common approaches to grouping students include an approach where students are grouped based on their aptitude and a cross-sectional approach where equal groups are formed and comprised of students of varied aptitudes. Instructional Grouping Theory is the study of how selection strategies impact the learning of group members. Using a…

Infant research is often underpowered, undermining the robustness and replicability of our findings. Improving the reliability of infant studies offers a solution for increasing statistical power independent of sample size. Here, we discuss two senses of the term reliability in the context of infant research: reliable (large) effects and reliable…

In this study, various statistical indexes of agreement were calculated using empirical data from a group of evaluators (n = 45) of early childhood teachers. The group of evaluators rated ten fictitious teacher profiles using the North Carolina Teacher Evaluation Process (NCTEP) rubric. The exact and adjacent agreement percentages were calculated…

Bounded continuous data are encountered in many applications of item response theory, including the measurement of mood, personality, and response times and in the analyses of summed item scores. Although different item response theory models exist to analyze such bounded continuous data, most models assume the data to be in an open interval and…

For ease of instruction in the classroom, the one-way analysis of variance F statistic is rewritten in terms of pairwise differences in individual sample means instead of differences of individual sample means from the overall sample mean. Likewise, the Kruskal-Wallis statistic may be rewritten in terms of pairwise differences in individual…

Artificial neural networks (NN) can help researchers estimate propensity scores for quasi-experimental estimation of treatment effects because they can automatically detect complex interactions involving many covariates. However, NN is difficult to implement due to the complexity of choosing an algorithm for various treatment levels and monitoring…

Many nature of science (NOS) studies have demonstrated that teachers can improve their understandings of NOS with explicit and reflective instruction; however, the field has relied heavily on qualitative methods. Because qualitative studies can be difficult to compare across studies and contexts, our study sought to determine if a quantitative…

Currently the design standards for single case experimental designs (SCEDs) are based on validity considerations as prescribed by the What Works Clearinghouse. However, there is a need for design considerations such as power based on statistical analyses. We compute and derive power using computations for (AB)[superscript k] designs with multiple…

Applied researchers faced with limited resources can utilize planned missing designs by incorporating missing data into their research design to collect more and higher quality data compared to a conventional experimental design. Two-method measurement planned missing designs (TMM-PMD) are a type of planned missing design whereby researchers…

Combining observational and experimental data for causal inference can improve treatment effect estimation. However, many observational data sets cannot be released due to data privacy considerations, so one researcher may not have access to both experimental and observational data. Nonetheless, a small amount of risk of disclosing sensitive…

Racism impacts the lives of students who identify as Black, Indigenous, or People of Color (BIPOC) in a myriad of ways. It is important that future teachers go beyond individual acts of racism to understand how racism operates as a system. To this end, we designed and implemented a statistical investigation with 13 preservice teachers using real…

In the era of the "reproducibility crisis" and the "P-value controversy" new ways of presentation and interpretation of the results of a meta-analysis are desirable. One suggestion that has been made for single studies almost six decades ago and taken up now and then is the P-value function. For a given outcome, this function…

Statistical methods are increasingly being used to integrate findings from the ever-expanding universe of empirical research. Meta-analysis encompasses various techniques for synthesizing summary statistics, and mega-analysis pools raw data across studies. This paper offers an introduction to meta-analysis and mega-analysis that complements the…