How can we test whether state-of-the-art generative models, such as Blender and GPT-3, are good AI teachers, capable of replying to a student in an educational dialogue? Designing an AI teacher test is challenging: although evaluation methods are much-needed, there is no off-the-shelf solution to measuring pedagogical ability. This paper reports…

This quasi-experimental research aims to investigate the effects of intensive and extensive recasts on learning the English simple present singular -s. In line with this purpose, firstly 56 beginner level learners were non-randomly assigned to three different groups: written intensive recast (n = 20), written extensive recast (n = 17), and control…

The argument-based approach to validation involves two steps; specification of the proposed interpretations and uses of the test scores as an interpretive argument, and the evaluation of the plausibility of the proposed interpretive argument. More ambitious interpretations and uses tend to involve an extended network of inferences and assumptions…

Martha Urioste tells a compelling history of public sector Montessori in Denver, from its socially charged beginnings in the 1960s to 2009. She played a huge role in creating the first public Montessori school in Colorado that was launched as an integration model with bilingual classes and staff. This inspirational story begins with her early…

In the present study, university students were presented a scenario in which a married couple was struggling in their marriage. These students were asked how likely it is that they would stay in a difficult marriage like the one described in the scenario. Each student also completed Schwartz's (2004) Maximization Scale. High scorers on this scale…

Past researchers of education candidates have remarked that educational administration (EA) has attracted students of below-average verbal, quantitative, and analytic scores as measured by standardized tests. This is a two-fold problem: (1) Principals with high overall academic ability are more likely to be viewed as leaders in…

A survey was conducted of a sample of 716 well-qualified students who sent their American College Testing Program (ACT) scores to the University of Michigan but subsequently did not apply for admission. Telephone interviews resulted in 574 completed interviews, for an 80% response rate. The interview instrument elicited information regarding the…

Possible interpretive arguments for licensure and certification test scores are examined, and evidence to support each type of argument is analyzed. Particular attention is given to a competency-based argument that involves a sequence of inferences from test scores to statements about competence and then to conclusions about expected performance.…

An agenda for research on standards for professional licensure and certification examinations is outlined. The relationship between scores and standards is explored, and major classes of standards are described. Topics for future research are discussed in the areas of judgment basis, process efficiency, group effects, and content and expertise…

This study compared the analytic scores on an essay test of clinical judgment generated by nonphysician readers with those of physician readers. Generalizability theory was applied to determine the number of essays and scorers required to reach specified levels of reproducibility. Results suggested that with training nonphysicians could…

Most training programs in education and psychology focus on classical test theory techniques for assessing score dependability. This paper discusses generalizability theory and explores its concepts using a small heuristic data set. Generalizability theory subsumes and extends classical test score theory. It is able to estimate the magnitude of…

A part-form methodology was used to study the effect of varying degrees of multidimensionality on the consistency of pass/fail classification decisions obtained from simulated unidimensional item response theory (IRT) based licensure examinations. A control on the degree of form multidimensionality permitted an assessment throughout the range of…

This paper considers the problem of validating placement procedures or, more precisely, of determining their educational appropriateness. At issue is determining whether a test score serves the particular educational function it was designed to serve (for example, course placement), and whether it does so in an economical way. These determinations…

Career decision making is an important aspect of life. At some point one must decide what type of job to apply for or whether to seek education for a specific career. Many individuals struggle with these decisions. Unfortunately, multidimensional measures of uncertainty are needed in providing guidance to such persons, because it appears the…

In recent years, standardized test scores have assumed an increasingly important role in educational assessment. As the stakes involved in testing become higher, educators are forced to make decisions regarding the ethics involved in standardized test preparation and administration. The ethical continuum of test preparation practices as formulated…