NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: EJ1202728
Record Type: Journal
Publication Date: 2018-Dec
Pages: 17
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-2330-8516
EISSN: N/A
The Impact of Aberrant Responses and Detection in Forced-Choice Noncognitive Assessment. Research Report. ETS RR-18-32
Kim, Sooyeon; Moses, Tim
ETS Research Report Series, Dec 2018
The purpose of this study is to assess the impact of aberrant responses on the estimation accuracy in forced-choice format assessments. To that end, a wide range of aberrant response behaviors (e.g., fake, random, or mechanical responses) affecting upward of 20%--30% of the responses was manipulated under the multi-unidimensional pairwise preference (MUPP) model with forced-choice format. Under each aberrance condition, we also computed the lz statistic (a common person-fit index) to determine how well this index can accurately detect aberrant responses without erroneously misidentifying honest examinees. Some aberrant behaviors (e.g., limited responses) were more problematic than others (e.g., omitting) in terms of true trait level recovery. The lz statistic associated with each dimension of the MUPP form led to higher detection rates than did the overall lz statistic. When the dimensional lz approach was used, however, many honest examinees were misclassified as outliers. The detection rates of lz differed as a function of aberrant response types.
Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A