ERIC Number: EJ1290766
Record Type: Journal
Publication Date: 2021-Apr
Pages: 30
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1076-9986
EISSN: EISSN-1935-1054
Validation Methods for Aggregate-Level Test Scale Linking: A Case Study Mapping School District Test Score Distributions to a Common Scale
Reardon, Sean F.; Kalogrides, Demetra; Ho, Andrew D.
Journal of Educational and Behavioral Statistics, v46 n2 p138-167 Apr 2021
Linking score scales across different tests is considered speculative and fraught, even at the aggregate level. We introduce and illustrate validation methods for aggregate linkages, using the challenge of linking U.S. school district average test scores across states as a motivating example. We show that aggregate linkages can be validated both directly and indirectly under certain conditions such as when the scores for at least some target units (districts) are available on a common test (e.g., the National Assessment of Educational Progress). We introduce precision-adjusted random effects models to estimate linking error, for populations and for subpopulations, for averages and for progress over time. These models allow us to distinguish linking error from sampling variability and illustrate how linking error plays a larger role in aggregates with smaller sample sizes. Assuming that target districts generalize to the full population of districts, we can show that standard errors for district means are generally less than 0.2 standard deviation units, leading to reliabilities above 0.7 for roughly 90% of districts. We also show how sources of imprecision and linking error contribute to both within- and between-state district comparisons within versus between states. This approach is applicable whenever the essential counterfactual question--"what would means/variance/progress for the aggregate units be, had students taken the other test?"--can be answered directly for at least some of the units.
Descriptors: Equated Scores, Validity, Methods, School Districts, Sample Size, Error of Measurement, Achievement Tests, Educational Testing, National Competency Tests, Grade 4, Grade 8, Reading Tests, Mathematics Tests, Standardized Tests
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://bibliotheek.ehb.be:2814
Publication Type: Journal Articles; Reports - Research
Education Level: Elementary Education; Grade 4; Intermediate Grades; Grade 8; Junior High Schools; Middle Schools; Secondary Education
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED)
Authoring Institution: N/A
Identifiers - Assessments and Surveys: National Assessment of Educational Progress; Measures of Academic Progress
IES Funded: Yes
Grant or Contract Numbers: R305D110018