NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
ERIC Number: ED601370
Record Type: Non-Journal
Publication Date: 2018
Pages: 226
Abstractor: As Provided
ISBN: 978-1-3922-0001-8
ISSN: EISSN-
EISSN: N/A
The Situational Judgment Test Validity Void: Describing Participant Response Processes
Wolcott, Michael David Lee
ProQuest LLC, Ph.D. Dissertation, The University of North Carolina at Chapel Hill
Situational judgment tests (SJTs) are used to measure important components of professional competence that cannot be assessed via traditional tests of knowledge and skills. Despite their increasing popularity, there is a significant gap in the validity evidence and research on the response process to support how SJTs measure their intended constructs. This study evaluated an SJT to examine: (1) the factors that influence the response process, (2) the role of experience in the response process, (3) the role of contextual features in the response process, and (4) whether individuals attempt to identify the construct being assessed. Thirty participants--15 students and 15 pharmacists--completed a 12-item SJT designed to measure empathy. Each participant engaged in an uninterrupted think-aloud interview while they completed an SJT followed by a cognitive interview that asked specific questions about their decision-making process. Results of the qualitative and quantitative analyses suggest that the SJT response processes include the complex integration of comprehension, retrieval, judgments, and response selections. In addition, job-specific knowledge and experiences comprised a significant portion of the retrieval process. Moreover, there was evidence that SJTs are highly contextual and that item characteristics such as setting, actors, or relationships can influence the response process. There was limited evidence to suggest individuals attempt to identify the construct being assessed. In summary, this study provides a comprehensive evaluation of the response process involved in SJTs using rigorous qualitative methodologies and it contributes to foundational steps to generate the validity evidence necessary to aid in score interpretation and future research. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://bibliotheek.ehb.be:2222/en-US/products/dissertations/individuals.shtml.]
ProQuest LLC. 789 East Eisenhower Parkway, P.O. Box 1346, Ann Arbor, MI 48106. Tel: 800-521-0600; Web site: http://bibliotheek.ehb.be:2222/en-US/products/dissertations/individuals.shtml
Publication Type: Dissertations/Theses - Doctoral Dissertations
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A