The problem of assessing whether experimental results can be replicated is becoming increasingly important in many areas of science. It is often assumed that assessing replication is straightforward: All one needs to do is repeat the study and see whether the results of the original and replication studies agree. This article shows that the…

The problem of assessing whether experimental results can be replicated is becoming increasingly important in many areas of science. It is often assumed that assessing replication is straightforward: All one needs to do is repeat the study and see whether the results of the original and replication studies agree. This article shows that the…

Researchers and policymakers in the US and beyond increasingly seek to identify teaching qualities that are associated with academic achievement gains for K-12 students through effectiveness studies. Yet teaching quality varies with academic content and social contexts, involves multiple participants, and requires a range of skills, knowledge, and…

The goal of this chapter is to recommend quality criteria to guide evaluators' selections of sampling designs when mixing approaches. First, we contextualize our discussion of quality criteria and sampling designs by discussing the concept of interpretive consistency and how it impacts sampling decisions. Embedded in this discussion are…

The purpose of this study was to examine the quality of quantitative articles published in the "Journal of Counseling & Development." Quality concerns arose in regard to omissions of psychometric information of instruments, effect sizes, and statistical power. Type VI and II errors were found. Strengths included stated research…

This study compared and contrasted two quantitative scholarly articles in relation to their research designs. Their designs were analyzed by the comparison of research references and research specific vocabulary to describe how various research methods were used. When researching and analyzing quantitative scholarly articles, it is imperative to…

A concern for generalization dominates quantitative research. For generalizability and repeatability, identification of sample size is essential. The present study investigates 90 qualitative master's theses submitted for the Primary and Secondary School Science and Mathematics Education Departments, Mathematic Education Discipline in 10…

Clear, coherent and trusted, this book is the perfect guide to the main social research methods in use today. The much anticipated Third Edition of Clive Seale's bestselling title further expands its coverage to provide an authoritative introduction to all of the social research methods used to analyze qualitative and quantitative data. Written by…

This study explores a strategy for improving the stability of variance component estimates when only small samples are available, using a series of small, less complex generalizability (G) study designs as a surrogate for a single large design. (Author/BW)

There is need for objective and reliable single-case research (SCR) results in the movement toward evidence-based interventions (EBI), for inclusion in meta-analyses, and for funding accountability in clinical contexts. Yet SCR deals with data that often do not conform to parametric data assumptions and that yield results of low reliability. A…

Randomization should be employed whenever possible but the use of carefully controlled nonrandomized data to estimate causal effects is a reasonable and necessary procedure in many cases. (Author/BJG)

A structural equation modeling approach to scale reliability evaluation can be employed to estimate generalizability theory indexes in settings where sampling of subjects and conditions is carried out. In one- and two-facet crossed designs, it is demonstrated how this method can be used to obtain estimates of relative generalizability…

Much of educational administration research suffers from lack of conformity between data analysis and research design, inattention to the test/retest stability of measurement instruments, and the use of statistics for nominal or interval data with ordinal data, random-sample statistics with nonrandom samples, and unnecessarily complex statistical…

Discusses, in the context of library evaluation, the major sources of untrustworthy information and describes what can be done to control them. The emphasis is on assessing the impact of changed operations. (JD)

The paper describes the small sample stability of least square estimates of variance components within the context of generalizability theory. Monte Carlo methods are used to generate data conforming to some selected multifacet generalizability designs to illustrate the sampling behavior of variance component estimates. (Author/CTM)