Combining methods from earlier content validity and more contemporary content alignment studies may allow a more complete evaluation of the meaning of test scores than if either set of methods is used on its own. This article distinguishes item relevance indices in the content validity literature from test representativeness indices in the…

In achievement testing there is typically a practical requirement that the set of items administered should be representative of some target content domain. This is accomplished by establishing test blueprints specifying the content constraints to be followed when selecting the items for a test. Sometimes, however, students give disengaged…

Although both content alignment and standard-setting procedures rely on content-expert panel judgements, only the latter employs discussion among panel members. This study employed a modified form of the Webb methodology to examine content alignment for twelve tests administered as part of the Massachusetts Comprehensive Assessment System (MCAS).…

This article reviews the history of the measure of socio-economic status in PISA and identifies theoretical underpinnings of the index of economic, social and cultural status (ESCS). It then highlights multiple changes in the instruments and scaling methods used by PISA over time, and suggests ways of resolving the tensions behind some of these…

Combining Bachman's (1990) conceptualization of content validity with Messick's (1989) unifying model of construct validity, this study attempted to fill the gap in researching content validation in the context of a university reading achievement test by (a) examining traditional content validity evidence and (b) analyzing the test scores to…

Background/Context: Adding global competency to the Organisation of Economic Cooperation and Development (OECD) Program of International Student Assessment (PISA) suite heralds the world's first large-scale attempt at gauging education systems' development of students' global competency. Given the contested definitions and more than 150 extant…

The use of assessment results to inform school accountability relies on the assumption that the test design appropriately represents the content and cognitive emphasis reflected in the state's standards. Since the passage of the Every Student Succeeds Act and the certification of accountability assessments through federal peer review practices,…

This article addresses test translation review in international test comparisons. We investigated the applicability of the theory of test translation error--a theory of the multidimensionality and inevitability of test translation error--across source language-target language combinations in the translation of PISA (Programme of International…

Gender differences in reading seem to increase throughout schooling and then decrease or even disappear with age, but the reasons for this are unclear. In this study, we explore whether differences in the way "reading literacy" is operationalised can add to our understanding of varying gender differences in international large-scale…

The COVID-19 pandemic has disrupted education across the globe leading countries to adapt how they administer and manage high-stakes examinations and large-scale learning assessments. This thematic review describes the measures that countries have taken, in terms of policies and practices, when learning assessments are disrupted by emergencies and…

This guide is designed to assist States, agencies, and/or facilities who work with youth who are neglected, delinquent, or at-risk (N or D). The information in the guide will benefit those who are (a) interested in implementing pre-posttests, (b) in the process of identifying an appropriate pre-posttest, or (c) ready to evaluate current testing…

This article investigates the persistent and change elements of educational testing and assessment from 1920 to the present day. I show by examining the addresses and texts of American Educational Research Association presidents a continuing focus on schools, from early experiments and development up through applications in accountability systems.…

This report will determine to what degree the AP Physics 1 and 2 and AP Calculus AB and BC frameworks are aligned with the Trends in International Mathematics and Science Study (TIMSS) Advanced Physics and Mathematics frameworks. This will enable an exploration of any differences in content coverage and levels of complexity, and will set the stage…

Background: In 2011 the Progress in International Reading Literacy Study (PIRLS) and the Trends in International Mathematics and Science Study (TIMSS) were conducted at fourth grade in a number of participating countries with a shared representative sample. In this article we investigate whether there are multidimensional proficiency patterns…

Background: While rotated test booklets have been employed in large-scale assessments to increase the content coverage of the assessments, rotation has not yet been applied to the context questionnaires administered to respondents. Methods: This paper describes the development of a methodology that uses rotated context questionnaires in…