ERIC Number: EJ968403
Record Type: Journal
Publication Date: 2012-Jun
Pages: 26
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1076-9986
EISSN: N/A
Using a Theorem by Andersen and the Dichotomous Rasch Model to Assess the Presence of Random Guessing in Multiple Choice Items
Andrich, David; Marais, Ida; Humphry, Stephen
Journal of Educational and Behavioral Statistics, v37 n3 p417-442 Jun 2012
Andersen (1995, 2002) proves a theorem relating variances of parameter estimates from samples and subsamples and shows its use as an adjunct to standard statistical analyses. The authors show an application where the theorem is central to the hypothesis tested, namely, whether random guessing to multiple choice items affects their estimates in the Rasch model. Taking random guessing to be a function of the difficulty of an item relative to the proficiency of a person, the authors describe a method for creating a subsample of responses, which is least likely to be affected by guessing. Then using Andersen's theorem, the authors assess the difference in difficulty estimates between responses from the whole sample and the subsample for each item. To demonstrate the effectiveness of the procedure, data are simulated according to a class of models in which random guessing is a function of the proficiency of a person relative to the difficulty of an item. The procedure is also applied to an empirical data set from Raven's Advanced Progressive Matrices, with the results indicating that guessing is present in a substantial number of items. It is noted that one especially important application in which estimating the correct relative difficulty of items is required is where the items will form part of an item bank and where on subsequent occasions the items will be administered interactively. In this case, items too difficult for a person are not administered and therefore unlikely to attract random guessing. (Contains 3 tables and 6 figures.)
Descriptors: Test Items, Item Response Theory, Multiple Choice Tests, Guessing (Tests), Item Banks, Difficulty Level, Mathematical Logic, Validity
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://bibliotheek.ehb.be:2814
Publication Type: Journal Articles; Reports - Descriptive
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Raven Advanced Progressive Matrices
Grant or Contract Numbers: N/A