ERIC Number: ED606154
Record Type: Non-Journal
Publication Date: 2020-May
Pages: 10
Abstractor: ERIC
ISBN: N/A
ISSN: N/A
EISSN: N/A
Reliability of English Learners' Test Scores. Technical Brief
Moore, Joann L.; Li, Tianli; Lu, Yang
ACT, Inc.
The Every Student Succeeds Act requires that English Learners (ELs) are included in annual state testing (grades 3-8 and once in high school) and included in each state's accountability system disaggregated by subgroup to ensure that they receive the support they need to learn English, participate fully in their education experience, and graduate ready for college or career (US Department of Education, 2016). As more states are using the ACT® test as part of their accountability systems, one concern that educators and policymakers may have is whether the scores of English Learners (ELs) are valid and reliable indicators of their actual academic achievement level. This research brief addresses the following research questions: (1) How does the reliability of ELs' ACT scores compare to that of non-ELs?; (2) How does the reliability of ACT scores for ELs compare to the reliability of other standardized assessment scores?; and (3) How does classification consistency and differential item functioning (DIF) analyses provide additional evaluative information about score validity? Findings show that limited English proficiency can be a source of measurement error when ELs are assessed in the English language such that students who have difficulty comprehending and responding accurately to test content may not be able to fully demonstrate their true achievement level. The reliability of ELs' scores on the ACT is comparable to that seen in other assessments and is sufficiently high that it does not by itself raise concerns about the validity of their scores. The study did not find evidence of DIF for ELs, and classification consistency analyses revealed similar agreement rates for ELs and non-ELs. These findings are encouraging and suggest that item characteristics are not introducing additional bias that would raise concerns about score validity for ELs.
Descriptors: College Entrance Examinations, Scores, English Language Learners, Accountability, Standardized Tests, Classification, Test Bias, Test Validity, Test Reliability, Limited English Speaking, Academic Achievement, Error of Measurement, Elementary Secondary Education, Postsecondary Education
ACT, Inc. 500 ACT Drive, P.O. Box 168, Iowa City, IA 52243-0168. Tel: 319-337-1270; Web site: http://www.act.org
Publication Type: Reports - Research
Education Level: Higher Education; Postsecondary Education; Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: ACT, Inc.
Identifiers - Assessments and Surveys: ACT Assessment
Grant or Contract Numbers: N/A