We address two persistent challenges in large-scale assessments of the Next Generation Science Standards: (a) the validity of score interpretations that target the standards broadly and (b) how to structure claims for assessments of this complex domain. The NGSS pose a particular challenge for specifying claims about students that evidence from…

The use of computer-based tests (CBTs), for both formative and summative purposes, has greatly increased over the past years. One major advantage of CBTs is the easy integration of multimedia. It is unclear, though, how to design such CBT environments with multimedia. The purpose of the current study was to examine whether guidelines for designing…

ACT requested assistance from the National Center for the Improvement of Educational Assessment (Center for Assessment) to investigate declines of scores for states administering the ACT to its 11th grade students in 2018. This request emerged from conversations among state leaders, the Center for Assessment, and ACT in trying to understand the…

Reading comprehension is often treated as a multidimensional construct. In many reading tests, items are distributed over reading process categories to represent the subskills expected to constitute comprehension. This study explores (a) the extent to which specified subskills of reading comprehension tests are conceptually conceivable to…

The OECD Programme for International Student Assessment (PISA) examines what students know in reading, mathematics and science, and what they can do with what they know. It provides the most comprehensive and rigorous international assessment of student learning outcomes to date. Results from PISA indicate the quality and equity of learning…

Test scores matter these days. Test-takers want to understand how they performed, and test score reports, particularly those for individual examinees, are the vehicles by which most people get the bulk of this information. Historically, score reports have not always met the examinees' information or usability needs, but this is clearly changing…

This study compared an expert-based cognitive model of domain mastery with student-based cognitive models of task performance for Integrated Algebra. Interpretations of student test results are limited by experts' hypotheses of how students interact with the items. In reality, the cognitive processes that students use to solve each item may be…

Hooker, Finkelman, and Schwartzman ("Psychometrika," 2009, in press) defined a paradoxical result as the attainment of a higher test score by changing answers from correct to incorrect and demonstrated that such results are unavoidable for maximum likelihood estimates in multidimensional item response theory. The potential for these results to…

To achieve perpetually better test results each year as mandated by the No Child Left Behind Act (NCLB), teachers in successful schools such as Leroy Anderson Elementary in San Jose, California, will "try anything" to raise scores, as the school's principal stated in an interview with "The San Jose Mercury News." In schools…

In this study, differential item functioning (DIF) methods utilizing 14 different matching variables were applied to assess DIF in the constructed-response (CR) items from 6 forms of 3 mixed-format tests. Results suggested that the methods might produce distinct patterns of DIF results for different tests and testing programs, in that the DIF…

The release of School Grades by the State of Florida in the late summer of 2010 created quite a stir. Over 500 elementary schools all over the state dropped at least one grade level. Among those schools that experienced drops, the Reading Gains and Mathematics Gains components seemed especially low. Some of the historically best performing schools…

The purpose of this paper is to propose an applied model for the assessment of bilingual children's language abilities with standardised tests. We discuss the purposes of such tests, especially in relation to vocabulary knowledge, and potential applications of test results for each of those purposes. The specific case to be examined here is that…

Driven largely by calls for accountability, the use of large-scale testing is expanding in terms of the number and purposes of testing programmes. At the same time, financial constraints have resulted in attempts to reduce the lengths of such examinations. An examination of the 1994/1995 and 1995/1996 British Columbia Scholarship programme…

Describes an analytic method for aiding in the generation of subscores that characterize the deep structure of tests and derives a procedure for estimating scores for those scales that are more statistically stable than subscores composed solely of items contained on that scale. Used data from a Praxis administration (9,278 examinees) to show the…

Attempted to improve the measurement quality of a complex performance assessment through principled assessment design using the example of the National Board for Professional Teaching Standards Early Childhood/Generalist examination. All indexes examined improved after revisions were made. Results show the importance of attention to assessment…