ERIC Number: EJ1397090
Record Type: Journal
Publication Date: 2023
Pages: 14
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1536-6367
EISSN: EISSN-1536-6359
Person Misfit and Person Reliability in Rating Scale Measures: The Role of Response Styles
Zou, Tongtong; Bolt, Daniel M.
Measurement: Interdisciplinary Research and Perspectives, v21 n3 p167-180 2023
Person misfit and person reliability indices in item response theory (IRT) can play an important role in evaluating the validity of a test or survey instrument at the respondent level. Prior empirical comparisons of these indices have been applied to binary item response data and suggest that the two types of indices return very similar results. In this paper, however, we demonstrate an important applied distinction between these methods when applied to polytomously-scored rating scale items, namely their varying sensitivities to response style tendencies. Using several empirical datasets, we illustrate settings in which these indices are in one case highly correlated and in two other cases weakly correlated. In the datasets showing a weak correlation between indices, the primary distinction appears due to the effects of response style behavior, whereby respondents whose response styles are less common (e.g. a disproportionate selection of the midpoint response) are found to misfit using Drasgow et al's l[subscript z] person misfit index, but often show high levels of reliability from a person reliability perspective; just the opposite frequently occurs for respondents that over-select the rating scale extremes. It is suggested that person misfit reporting should be supplemented with an evaluation of person reliability to best understand the validity of measurement at the respondent level when using IRT models with rating scale measures.
Descriptors: Item Response Theory, Rating Scales, Response Style (Tests), Measurement, Test Reliability, Test Validity, Goodness of Fit, Personality Traits, Extraversion Introversion
Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A