ERIC Number: ED652360
Record Type: Non-Journal
Publication Date: 2020
Pages: 275
Abstractor: As Provided
ISBN: 979-8-5699-0530-0
ISSN: N/A
EISSN: N/A
Assess Robustness of the Rasch Mixture Model to Detect Differential Item Functioning -- A Monte Carlo Study
Jinjin Huang
ProQuest LLC, Ph.D. Dissertation, University of Denver
Measurement invariance is crucial for an effective and valid measure of a construct. Invariance holds when the latent trait varies consistently across subgroups; in other words, the mean differences among subgroups are only due to true latent ability differences. Differential item functioning (DIF) occurs when measurement invariance is violated. There are two kinds of traditional tools for DIF detection: non-parametric methods and parametric methods. Mantel Haenszel (MH), SIBTEST, and standardization are examples of non-parametric DIF detection methods. The majority of parametric DIF detection methods are item response theory (IRT) based. Both non-parametric methods and parametric methods compare differences among subgroups categorized by observed covariates such as gender and grade. As a result, the differences within unobserved subgroups are likely to be neglected. The Rasch mixture model (RMM), a combination of the Rasch model and mixture model, is an alternative for extracting the latent class (LC) from summarizing similar identities of underlying latent traits. DIF can be calculated among LCs based on the differences among mean item difficulties for each LC. The purpose of this study was to examine the robustness of the RMM in detecting DIF through manipulating five variables: number of items (i.e., test length, 2 levels), proportion of DIF (3 levels), LC structure (2 levels), group size (2 levels) and DIF type (2 levels), which yields 2*3*2*2*2 = 48 scenarios. A sample size of 3,000 was used for each replication of each scenario. The robustness of the RMM on detecting DIF was assessed from two perspectives: latent class structure recovery and parameter recovery. One hundred replications per scenario were used for LC structure recovery and 200 replications per scenario were used for parameter recovery. The main and interactions effects of five manipulated factors on LC structure recovery and parameter recovery were examined by conducting factorial analysis of variance (ANOVA). Both AIC and BIC showed a conservative pattern on LC structure recovery in which the recovered LCs did not match the true structure perfectly or even in the majority of cases. That is, it was rare that the correct latent structure was recovered at 100%. For classifier recovery, all five manipulated factors showed effect sizes that were medium or larger except DIF type ([eta-squared] < 0.06), and there were two medium effect size interactions for classifier recovery, and they were number of items by group size interaction ([eta-squared] = 0.10) and LC structure by group size interaction ([eta-squared] = 0.13). There were three main and three interaction effects of the five manipulated factors on DIF recovery ([eta-squared] > 0.06) and they were effects of number of items, proportion of DIF items, LC structure, number of items by LC structure interaction, proportion of DIF items by LC structure interaction, and DIF type by LC structure interaction. Among these effects, group size ([eta-squared] = 0.45) had the strongest effect on classifier recovery and LC structure ([eta-squared] = 0.86) had the strongest effect on DIF recovery. It is recommended for practitioners to have close group sizes for latent classes, 20% to 40% proportion of DIF items, and a LC structure close to a two LC structure, to determine DIF using an RMM. Both AIC and BIC are not suggested as model selection methods in DIF detection using the RMM. Instead the Cressie-Read statistic can be an option for choosing the correct number of latent classes from observed response patterns as the Cressie-Read statistic includes statistical tests rather than using likelihood ratios. A practitioner can identify DIF and its direction through calculating the item difficulty difference [delta]b between two latent classes. It can be considered as no item DIF for using the RMM method when [delta]b < 0.3, small DIF when 0.3 [less than or equal to] [delta]b < 0.9, medium DIF when 0.9 [less than or equal to] [delta]b < 1.5, and large DIF when [delta]b [greater than or equal to] 1.5.Finding more reliable model selection indices for the RMM on DIF detection, increasing the efficiency of simulation, and including a single latent class structure as a comparison are directions for future study. The number of replications used in this study is recommended for practitioners who want to conduct simulation studies using the Rasch mixture model. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://bibliotheek.ehb.be:2222/en-US/products/dissertations/individuals.shtml.]
Descriptors: Robustness (Statistics), Item Response Theory, Test Items, Item Analysis, Monte Carlo Methods, Error of Measurement, Nonparametric Statistics, Statistical Analysis, Statistical Studies, Identification, Sample Size, Replication (Evaluation)
ProQuest LLC. 789 East Eisenhower Parkway, P.O. Box 1346, Ann Arbor, MI 48106. Tel: 800-521-0600; Web site: http://bibliotheek.ehb.be:2222/en-US/products/dissertations/individuals.shtml
Publication Type: Dissertations/Theses - Doctoral Dissertations
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A