This paper summarizes the findings from a panel of assessment experts on diagnostic assessments and their role in helping educators and parents support student learning. Teachers, schools, and school systems will face unprecedented challenges when schools eventually reopen after pandemic-related closures. One of the central challenges will be…

In the 2018 meta-analysis of "Educational Psychology Review" entitled "Null effects of perceptual disfluency on learning outcomes in a text-based educational context" by Xie, Zhou, and Liu, we identify some errors and inconsistencies in both the methodological approach and the reported results regarding coding and effect sizes.…

This position paper introduces the idea of cognicy, the foundational ability to think and understand in a process that decouples cognitive processes from their tangible outcomes. Generative artificial intelligence (AI) can produce output often nearly indistinguishable from a human product, which presents a problem for educational assessment.…

I argue that teaching evaluation tools (TETs) may function as ethical codes (ECs), and answer certain demands that ECs cannot sufficiently fulfill. In order to be viable, an EC related to the teaching profession must assume a different form, and such a form is already present in several of the contemporary TETs. The TET matrix form allows for…

It is nearly impossible today to discuss education without considering testing and evaluation. In the opening article of this volume, professors Baird, Andrich, Hopfenbeck, and Stobart (2017) lay out the challenge of putting learning and teaching first, tracing the development of the science of educational assessment and raising the question…

George Grob presented the fifth and final Eleanor Chelimsky Forum address at the 2017 annual meeting of the Eastern Evaluation Research Society. In this commentary, I respond to several points that George raises in the "American Journal of Evaluation" paper based on that address. An overarching theme of my comments involves the potential…

Confirming multisite studies presents an ongoing challenge for researchers based in England. Those wishing to set up a multisite study must undergo local Research & Development review, in addition to ethical review by a Research Ethics Committee. However, both the documentation required for multisite review and the length of time it takes to…

The target paper, "Rethinking Traditional Methods of Survey Validation" (Andrew Maul), raises some interesting critical ideas, both old and new, about the validation of self-report surveys. As indicated by Dr. Maul, recent policy initiatives in the United States (e.g., ESSA) have led to a demand for assessments of…

In this article, Dylan William states that the central claim in Baird, et al.'s piece is that if theories of assessment take into account theories of learning, assessments will somehow be more valid, and some of the more egregious effects of assessment on learning will be ameliorated. William responds to this claim by arguing that it seems…

In this article, Hartley notes the difficulties of using questionnaires to assess the efficiency of new instructional methods and highlights nine issues that researchers must consider. Hartley continues the discussion about the use of questionnaires and suggests that psychology teachers can help improve the teaching of psychology by drawing…

This letter documents some problems in Ancaiani et al. (2015). Namely the evaluation of concordance, based on Cohen's kappa, reported by Ancaiani et al. was not computed on the whole random sample of 9,199 articles, but on a subset of 7,597 articles. The kappas relative to the whole random sample were in the range 0.07-0.15, indicating an…

The challenge of "educational" assessments--assessments that advance the purposes of learning and instruction--is to provide useful information regarding students' progress towards the goals of instruction in ways that are reliable and not idiosyncratic. In this commentary, the author indicates that the challenges are actually more…

The return on research investment resulting from new breakthrough scientific discoveries may be decreasing over time due to the law of diminishing returns, the relative decrease of research funding in terms of purchasing power parity, and various activities gaming the system. By altering the grant-review process, the scientific community may…

Aguirre-Urreta, Rönkkö, and Marakas' (2016) paper in "Measurement: Interdisciplinary Research and Perspectives" (hereafter referred to as ARM2016) is an important and timely piece of scholarship, in that it provides strong analytic support to the growing theoretical literature that questions the underlying ideas behind causal and…

In this commentary, Mary James highlights two problems she deemed critical during her work exploring the relationships between assessment and learning in theory and practice. First, efforts to improve assessment for learning were not always successful either in improving performance or in other ways. Second, and this may be a reason for the first…