ERIC Number: EJ998683
Record Type: Journal
Publication Date: 2013
Pages: 22
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0895-7347
EISSN: N/A
Item Difficulty and Interviewer Knowledge Effects on the Accuracy and Consistency of Examinee Response Processes in Verbal Reports
Leighton, Jacqueline P.
Applied Measurement in Education, v26 n2 p136-157 2013
The Standards for Educational and Psychological Testing indicate that multiple sources of validity evidence should be used to support the interpretation of test scores. In the past decade, examinee response processes, as a source of validity evidence, have received increased attention. However, there have been relatively few methodological studies of the accuracy and consistency of examinee response processes as measured by verbal reports in the context of educational measurement. The objective of the current study was to investigate the accuracy and consistency of examinee response processes--as measured by verbal reports--as a function of varying interviewer and item variables in a think aloud interview within an educational measurement context. Results indicate that the accuracy of responses may be undermined when students perceive the interviewer to be an expert in the domain. Further, the consistency of response processes may be undermined when items that are too easy or difficult are used to elicit reports. The implications of these results for conducting think-aloud studies are explored. (Contains 3 figures and 9 tables.)
Descriptors: Psychological Testing, Standards, Interviews, Protocol Analysis, Standardized Tests, Verbal Communication, Item Response Theory, Measurement, Accuracy, Validity, Reliability, Educational Research, Outcome Measures, Experimenter Characteristics, Student Attitudes, High School Students, Grade 12
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: Grade 12; High Schools
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A