NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
ERIC Number: ED645550
Record Type: Non-Journal
Publication Date: 2022
Pages: 177
Abstractor: As Provided
ISBN: 979-8-8340-7614-8
ISSN: N/A
EISSN: N/A
Posterior Predictive Model Checking of the Hierarchical Rater Model
Nnamdi Chika Ezike
ProQuest LLC, Ph.D. Dissertation, University of Arkansas
Fitting wrongly specified models to observed data may lead to invalid inferences about the model parameters of interest. The current study investigated the performance of the posterior predictive model checking (PPMC) approach in detecting model-data misfit of the hierarchical rater model (HRM). The HRM is a rater-mediated model that incorporates components of the polytomous item response theory (IRT) model, such as the partial credit model (PCM) and generalized partial credit model (GPCM), at the second level of the hierarchy, to model examinees' responses to performance assessments. To date, the HRM has not been rigorously evaluated using PPMC techniques. Monte Carlo simulations were employed to explore the effectiveness of 13 discrepancy measures in detecting model-data misfit of the HRM. Misfits were assessed at the test-, item-, and rater-level. Using the HRM-GPCM, data were generated by varying the rating design (fully-crossed and spiral), proportion of aberrant raters (no rater effects and 25% of the raters with rater effects), and number of examinees (250 and 500). Data generated were analyzed using the HRM-PCM and HRM-GPCM with eight raters and four items. Type I error and power rates were computed for each discrepancy measure. The results indicate that the standard deviation of the total score was the only useful discrepancy measure at the test level. Furthermore, the item-total correlation and odds ratio were found to be powerful in detecting misspecification of the HRM-PCM at the item level. Of the three rater-level discrepancy measures, only the score-estimate correlation and rater-total correlation were adequate in detecting the misfit of the HRM-PCM. The performance of the discrepancy measures in detecting misfit of HRM-PCM differed by the magnitude of the item discrimination parameters. The impact of the simulation factors on detecting misfit of the HRM-PCM and implications are further discussed. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://bibliotheek.ehb.be:2222/en-US/products/dissertations/individuals.shtml.]
ProQuest LLC. 789 East Eisenhower Parkway, P.O. Box 1346, Ann Arbor, MI 48106. Tel: 800-521-0600; Web site: http://bibliotheek.ehb.be:2222/en-US/products/dissertations/individuals.shtml
Publication Type: Dissertations/Theses - Doctoral Dissertations
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A