ERIC Number: EJ1366810
Record Type: Journal
Publication Date: 2022-Dec
Pages: 22
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: EISSN-2330-8516
Influence of Selected-Response Format Variants on Test Characteristics and Test-Taking Effort: An Empirical Study. Research Report. ETS RR-22-01
Guo, Hongwen; Rios, Joseph A.; Ling, Guangming; Wang, Zhen; Gu, Lin; Yang, Zhitong; Liu, Lydia O.
ETS Research Report Series, Dec 2022
Different variants of the selected-response (SR) item type have been developed for various reasons (i.e., simulating realistic situations, examining critical-thinking and/or problem-solving skills). Generally, the variants of SR item format are more complex than the traditional multiple-choice (MC) items, which may be more challenging to test takers and thus may discourage their test engagement on low-stakes assessments. Low test-taking effort has been shown to distort test scores and thereby diminish score validity. We used data collected from a large-scale assessment to investigate how variants of the SR item format may impact test properties and test engagement. Results show that the studied variants of SR item format were generally harder and more time consuming compared to the traditional MC item format, but they did not show negative impact on test-taking effort. However, item position had a dominating influence on nonresponse rates and rapid-guessing rates in a cumulative fashion, even though the effect sizes were relatively small in the studied data.
Descriptors: Test Format, Test Wiseness, Test Items, Item Response Theory, Difficulty Level, Scores, Effect Size, College Students, Foreign Countries, Reaction Time
Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: United States; Canada
Grant or Contract Numbers: N/A