This study investigated test-taking engagement on the Spring 2019 administration of a large-scale state summative assessment. Through the identification of rapid-guessing behavior -- which is a validated indicator of disengagement -- the percentage of Grade 8 test events with meaningful amounts of rapid guessing was 5.5% in mathematics, 6.7% in…

Reading comprehension tests involve not only reading passages and answering questions but also making choices such as the test-taking strategy to use, whether to search passages for answers, and where to begin searching. We examined the associations between student characteristics, passage type, test-taking strategies, and students' time searching…

Differential Item Function (DIF) analysis is commonly employed to examine potential bias produced by a test item. Since its introduction DIF analyses have focused on potential bias related to broad categories of oppression, including gender, racial stratification, economic class, and ableness. More recently, efforts to examine the effects of…

As more states move to universal computer-based assessments, an emergent issue concerns the effect that device type might have on student results. Although, several research studies have explored device effects, most of these studies focused on the differences between tablets and desktops/laptops. In the current study, we distinguish between…

Effectively evaluating the credibility and accuracy of multiple sources is critical for college readiness. We developed 24 source evaluation tasks spanning four predicted difficulty levels of a hypothesized learning progression (LP) and piloted these tasks to evaluate the utility of an LP-based approach to designing formative literacy assessments.…

This systematic review examines empirical research about students' motivation for NAEP in grades 4, 8, and 12 using multiple motivation constructs, including effort, value, and expectancy. Analyses yielded several findings. First, there are stark differences in the perceived importance of doing well on NAEP among students in grades 4 (86%), 8…

As students return to in-person instruction in the 2021-2022 school year, local education agencies (LEAs) must develop resource allocation strategies to support schools in need. Federal programs have provided resources to support restart and recovery. However, there is little consensus on how LEAs can target resources to support those schools most…

This paper explores ways of capturing teachers' formative assessment behaviors in Philippine classrooms through an observation tool. Early versions of the tool were structured using the 'Elicit-Student response-Recognize-Use' ESRU model. To account for the practices observed in the classroom, the observation tool was resituated to focus on Elicit…

Writing from source text is critical for developing college-and-career readiness because it is required in advanced academic environments and many vocations. Scenario-based assessment (SBA) represents one approach to measuring this ability. In such assessment, the scenario presents an issue that the student is to read and write about. Before…

Using two states' grades 3 through 8 state assessment databases, this study documents the extent to which students were assigned testing accommodations for ELA or mathematics in only one of two consecutive years. The percentage of students with disabilities who were assigned accommodations in the current year only or in the prior year only in a…

High-quality measures of instructional practice are essential for research and evaluation of innovative instructional policies and programs. However, existing measures have generally proven inadequate because of cost and validity issues. This paper addresses two potential drawbacks of survey self-report measures: variation in teachers'…

We investigated the use of adaptive comparative judgment to evaluate the middle school student learning, engagement, and experience with the design process in an open-ended problem assigned in a technology and engineering education course. Our results indicate that the adaptive comparative judgment tool effectively facilitated the grading of the…

This study compared short-form constructed responses evaluated by both human raters and machine scoring algorithms. The context was a public competition on which both public competitors and commercial vendors vied to develop machine scoring algorithms that would match or exceed the performance of operational human raters in a summative high-stakes…

For the past decade, the accountability model associated with No Child Left Behind (NCLB) emphasized proficiency on end of year tests; with Every Student Succeeds Act (ESSA) the emphasis on proficiency within statewide testing programs, though now integrated with other measures of student learning, nevertheless remains a primary metric for…

In this article we explore the use of pictorial glossaries as an accommodation for English learners (ELs) with entry and emerging levels of English language proficiency. Drawing on survey responses from 98 middle school ELs and cognitive interviews with 10 of the survey participants, we examined the participants' preferences and experiences with…