ERIC Number: EJ1441175
Record Type: Journal
Publication Date: 2024
Pages: 18
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: EISSN-2602-3717
Assessing the Fairness of Mathematical Literacy Test in Indonesia: Evidence from Gender-Based Differential Item Function Analysis
Journal of Pedagogical Research, v8 n3 p191-208 2024
Conducting a fair test is important for educational research. Unfair assessments can lead to gender disparities in academic achievement, ultimately resulting in disparities in opportunities, wages, and career choice. Differential Item Function [DIF] analysis is presented to provide evidence of whether the test is truly fair, where it does not harm or benefit certain groups of students. For this reason, this study aims to assess the fairness of mathematics literacy tests from a gender perspective using three DIF analysis approaches, namely, the Cognitive Diagnostic Model [CDM], Classical Test Theory [CTT], and Item Response Theory [IRT], and to compare the results of the three approaches to examine the compatibility between them in identifying DIF effects. This study was included in quantitative descriptive research, and for the CDM approach, a retrofitting method (post-hoc analysis) was used. The sample in this study consists of Indonesian students who participated in the administration of PISA 2012 and were tested on Booklet 1, Booklet 3, Booklet 4, and Booklet 6. The Q-matrix used in this study consisted of 12 items and 11 attributes. The results of this study show that out of the 12 items analyzed, there are differences in findings between the CTT, IRT, and CDM approaches; the item with the largest DIF was found using the Raju Unsigned Area Measures method in IRT and the Wald Test from the CDM approach, while the item with the lowest DIF was found using the LRT method from the CDM approach; and there are three items that were simultaneously identified as DIF using the CTT, IRT, and CDM methods, namely PM923Q01, PM923Q03, and PM924Q02. Items PM923Q01 and PM923Q03 favor the group of male students, while item PM924Q02 favors the group of female students.
Descriptors: Foreign Countries, Test Bias, Item Response Theory, Test Theory, Gender Differences, Achievement Tests, Secondary School Students, Adolescents, Mathematics Tests, Mathematics Skills, Scores, Psychometrics, Gender Bias
Journal of Pedagogical Research. Duzce University, Faculty of Education, Konuralp Campus, 81620, Duzce, Turkey. e-mail: ijopr.editor@gmail.com; Web site: https://www.ijopr.com/
Publication Type: Journal Articles; Reports - Research
Education Level: Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Indonesia
Identifiers - Assessments and Surveys: Program for International Student Assessment
Grant or Contract Numbers: N/A