ERIC Number: EJ1331132
Record Type: Journal
Publication Date: 2022
Pages: 20
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1536-6367
EISSN: N/A
Handling Extreme Scores in Vertically Scaled Fixed-Length Computerized Adaptive Tests
Wyse, Adam E.; McBride, James R.
Measurement: Interdisciplinary Research and Perspectives, v20 n1 p1-20 2022
A common practical challenge is how to assign ability estimates to all incorrect and all correct response patterns when using item response theory (IRT) models and maximum likelihood estimation (MLE) since ability estimates for these types of responses equal -8 or +8. This article uses a simulation study and data from an operational K-12 computerized adaptive test (CAT) to compare how well several alternatives -- including Bayesian maximum a priori (MAP) estimators; various MLE based methods; and assigning constants -- work as strategies for computing ability estimates for extreme scores in vertically scaled fixed-length Rasch-based CATs. Results suggested that the MLE-based methods, MAP estimators with prior standard deviations of 4 and above, and assigning constants achieved the desired outcomes of producing finite ability estimates for all correct and all incorrect responses that were more extreme than the MLE values of students that got one item correct or one item incorrect as well as being more extreme than the difficulty of the items students saw during the CAT. Additional analyses showed that it is possible for some methods to exhibit changes in how much they differ in magnitude and variability from the MLE comparison values or the b values of the CAT items for all correct versus all incorrect responses and across grades. Specific discussion is given to how one may select a strategy to assign ability estimates to extreme scores in vertically scaled fixed-length CATs that employ the Rasch model.
Descriptors: Scores, Adaptive Testing, Computer Assisted Testing, Test Length, Maximum Likelihood Statistics, Item Response Theory, Bayesian Statistics, Elementary Secondary Education, Reading Tests
Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A