NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
ERIC Number: ED600445
Record Type: Non-Journal
Publication Date: 2019-Oct
Pages: 32
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
An IRT Mixture Model for Rating Scale Confusion Associated with Negatively Worded Items in Measures of Social-Emotional Learning. Working Paper
Bolt, Daniel M.; Wang, Yang Caroline; Meyer, Robert H.; Pier, Libby
Policy Analysis for California Education, PACE
We illustrate the application of mixture IRT [item response theory] models to evaluate the possibility of respondent confusion due to the negative wording of certain items on a social-emotional learning (SEL) assessment. Using actual student self-report ratings on four social-emotional learning scales collected from students in grades 3-12 from CORE districts in the state of California, we also evaluate the consequences of the potential confusion in biasing student- and school-level scores as well as correlational relationships between SEL and student-level variables. Models of both full and partial confusion are examined. Our results suggest that (1) rating scale confusion due to negatively-worded items does appear to be present; (2) the confusion is most prevalent at lower grade levels (3rd-5th); and (3) the occurrence of confusion is positively related to reading proficiency and ELL status, as anticipated, and bias estimates of SEL correlations with these student-level variables. For these reasons, we suggest future iterations of the SEL measures use only positively oriented items. To maintain measurement continuity, we suggest bias corrections based on the studied mixture model may be useful, although the precision of such corrections is sensitive to the nature of confusion (e.g., full versus partial).
Policy Analysis for California Education, PACE. 520 Galvez Mall, CERAS Room 401, Stanford, CA 94305-3001. Tel: 650-724-2832; Fax: 510-642-9148; e-mail: info@edpolicyinca.org; Web site: http://www.edpolicyinca.org
Publication Type: Reports - Research; Tests/Questionnaires
Education Level: Elementary Education; Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Stanford University, Policy Analysis for California Education (PACE)
Identifiers - Location: California
Grant or Contract Numbers: N/A