Methods to handle ordered-categorical indicators in latent variable interactions have been developed, yet they have not been widely applied. This article compares the performance of two popular latent variable interaction modeling approaches in handling ordered-categorical indicators: unconstrained product indicator (UPI) and latent moderated…

We compared students' performance on a paper-based test (PBT) and three computer-based tests (CBTs). The three computer-based tests used different test navigation and answer selection features, allowing us to examine how these features affect student performance. The study sample consisted of 9,698 fourth through twelfth grade students from across…

As the theme of the 2022 annual meeting of the American Education Research Association, cultivating equitable education systems has gained renewed attention amid an increasingly diverse society. However, systemic inequalities persist for traditionally underserved student populations. As a way to better address diverse students' needs, it is of…

Little is known empirically about intraclass correlations (ICCs) for multisite cluster randomized trial (MSCRT) designs, particularly in science education. In this study, ICCs suitable for science achievement studies using a three-level (students in schools in districts) MSCRT design that block on district are estimated and examined. Estimates of…

Students with significant cognitive disabilities present an assessment dilemma that centers on access and validity in large-scale testing programs. Typically, access is improved by eliminating construct-irrelevant barriers, while validity is improved, in part, through test standardization. In this article, one state's alternate assessment data…

This applied dissertation was designed to provide perceptual teacher data as well as summative testing data to educational leaders concerning the effects of implementing Investigations in Number, Data, and Space® (Investigations) in three Title I elementary school settings, two Title I schools, and one non-Title I school. Data collected during…

Accountability systems that measure student learning rather than student achievement have the potential to more accurately evaluate school quality. However, one methodological concern has remained surprisingly absent from discussions of value-added modeling. Standardized assessments that exhibit either positive or negative correlations between…

Despite policy efforts to encourage multiple measures of performance in newly developing teacher evaluation systems, practical constraints often result in evaluations based predominantly on formal classroom observations. Yet there is limited knowledge of how these observational measures relate to student achievement. This article leverages the…

It is often assumed that a vertical scale is necessary when value-added models depend upon the gain scores of students across two or more points in time. This article examines the conditions under which the scale transformations associated with the vertical scaling process would be expected to have a significant impact on normative interpretations…

Two ways of measuring change are presented and compared: A conventional "change score", defined as the difference between scores before and after an interim period, and a process-oriented approach focusing on detailed analysis of conceptually defined response patterns. The validity of the two approaches was investigated. Vocabulary…

The purpose of this article is to address a major gap in the instructional sensitivity literature on how to develop instructionally sensitive assessments. We propose an approach to developing and evaluating instructionally sensitive assessments in science and test this approach with one elementary life-science module. The assessment we developed…

If teacher value-added estimates (VAEs) are to be used as indicators of individual teacher performance in teacher evaluation and accountability systems, it is important to understand how much VAEs are affected by the data and model specifications used to estimate them. In this study we explored the effects of three conditions on the stability of…

This study examined the academic performance and preference of students with disabilities for two types of test administration conditions, computer-based testing (CBT) and pencil-and-paper testing (PPT). Data from a large-scale assessment program were used to examine differences between CBT and PPT academic performance for third to eleventh grade…

There are many advantages supporting the use of computers as an alternate mode of delivery for high stakes testing: cost savings, increased test security, flexibility in test administrations, innovations in items, and reduced scoring time. The purpose of this study was to determine if the use of computers as the mode of delivery had any…

The newly developed computerized Constructive Multiple-choice Testing system is introduced. The system combines short answer (SA) and multiple-choice (MC) formats by asking examinees to respond to the same question twice, first in the SA format, and then in the MC format. This manipulation was employed to collect information about the two…