Criteria are briefly proposed for final conclusions, research problems, and research hypotheses in quantitative research. Moreover, based on a proposed definition of applied and basic/general research, it is argued that (1) in applied quantitative research, while research problems are necessary, research hypotheses are unjustified, and that (2) in…

Missing data is a common challenge to any researcher in almost any field of research. In particular, human participants in research do not always respond or return for assessments leaving the researcher to rely on missing data methods. The most common methods (i.e., Multiple Imputation and Full Information Maximum Likelihood) assume that the…

In experimental research, internal validity refers to what extent researchers can conclude that changes in dependent variable (i.e. outcome) are caused by manipulations in independent variable. The causal inference permits researchers to meaningfully interpret research results. This article discusses (a) internal validity threats in social and…

Regression discontinuity design (RD) has been widely used to produce reliable causal estimates. Researchers have validated the accuracy of RD design using within study comparisons (Cook, Shadish & Wong, 2008; Cook & Steiner, 2010; Shadish et al, 2011). Within study comparisons examines the validity of a quasi-experiment by comparing its…

The sharp regression discontinuity design (RDD) has three key weaknesses compared to the randomized clinical trial (RCT). It has lower statistical power, it is more dependent on statistical modeling assumptions, and its treatment effect estimates are limited to the narrow subpopulation of cases immediately around the cutoff, which is rarely of…

In this chapter, the authors discuss the issue of research design in conducting inquiry on college impact and demonstrate the importance of longitudinal pretest-posttest designs in maximizing the internal validity of findings. They begin by discussing the strengths and weaknesses of different types of research design in the college impact…

A meta-analysis of outcomes from 32 studies investigating pretest effects was conducted. For all outcomes the average effect size was +.22, indicating an elevating effect of pretest on posttest. Duration of time between pre- and posttesting was also related to effect size. Researchers should continue to include pretest as a design variable.…

This paper delineates conditions under which the use of change scores will not produce misleading results, and may perhaps be preferable to other methods of analysis. The validity of change scores in randomized pretest-posttest designs is discussed along with situations where analysis of change scores should be used. (Author/GK)

Problems associated with the analysis of data collected using the Solomon Four Group Design are discussed. The design includes an experimental group and a control group that have been pretested and posttested, and an experimental and a control group that have been posttested only. A sample problem is approached in three different ways. First, the…

This review critiques the use of Lev Vygotsky's concept of the zone of proximal development (ZPD) in quantitative research that focuses on the role communication plays in learning. A study that makes claims in terms of the ZPD should include a pretest, a problem-solving activity, and a posttest. Without these minimal elements, researchers are not…

The purpose of this study was to determise the relative loss in statistical power of traditional methods of analysis when response-shift bias is present. Five methods of analysis (posttest scores only; postminus pretests; postminus retrospective pretest; postcovaried by pretest; postcovaried by retrospective pretest) were compared. (Author/BW)

The validity of various reliability-corrected procedures for adjusting for initial differences between groups in uncontrolled studies is established for subjects exhibiting linear fan-spread growth. The results are then extended to a nonlinear model of growth. (Author)

The validity of the equipercentile hypothesis of the Title I Evaluation and Reporting System norm-referenced evaluation model was examined using 3,224 seventh- and ninth-grade students. Findings from confidence interval procedures contradicted the equipercentile hypothesis. There was a pattern of large gains for students not receiving any special…

The biasing effects of a pretest on subsequent post-test results were investigated in two experimental studies. In general, the results argue for using designs without pretests. (Author/JKS)

The evaluation design, which prescribes when and from whom to gather data, is described as significant because it can lend credibility to evaluation results. The objectives of this booklet are to aid the evaluator in: choosing a design; putting it into operation; and analyzing and reporting the data. Examples include both formative and summative…