ERIC Number: EJ1421597
Record Type: Journal
Publication Date: 2024
Pages: 24
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: EISSN-2196-0739
Investigating Item Complexity as a Source of Cross-National DIF in TIMSS Math and Science
Qi Huang; Daniel M. Bolt; Weicong Lyu
Large-scale Assessments in Education, v12 Article 12 2024
Large scale international assessments depend on invariance of measurement across countries. An important consideration when observing cross-national differential item functioning (DIF) is whether the DIF actually reflects a source of bias, or might instead be a methodological artifact reflecting item response theory (IRT) model misspecification. Determining the validity of the source of DIF has implications for how it is handled in practice. We demonstrate a form of sensitivity analysis that can point to model misspecification induced by item complexity as a possible cause of DIF, and show how such a cause of DIF might be accommodated through attempts to generalize the IRT model for the studied item(s) in psychometrically and psychologically plausible ways. In both simulated illustrations and empirical data from TIMSS 2011 and TIMSS 2019 4th and 8th Grade Math and Science, we have found that using a form of proposed IRT model generalization can substantially reduce DIF when IRT model misspecification is at least a partial cause of the observed DIF. By demonstrating item complexity as a possible valid source of DIF and showing the effectiveness of the proposed approach, we recommend additional attention toward model generalizations as a means of addressing and/or understanding DIF.
Descriptors: Test Items, Item Response Theory, Test Bias, Test Validity, Difficulty Level, Psychometrics, Grade 4, Grade 8, Mathematics Tests, Science Tests, Generalization, Models, Achievement Tests, Elementary Secondary Education, Foreign Countries, Mathematics Achievement, Science Achievement, International Assessment
Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://bibliotheek.ehb.be:2123/
Publication Type: Journal Articles; Reports - Research
Education Level: Elementary Education; Grade 4; Intermediate Grades; Grade 8; Junior High Schools; Middle Schools; Secondary Education; Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Trends in International Mathematics and Science Study
Grant or Contract Numbers: N/A