ERIC Number: EJ1438132
Record Type: Journal
Publication Date: 2024-Mar
Pages: 24
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0272-2631
EISSN: EISSN-1470-1545
Estimating Reliability for Response-Time Difference Measures: Toward a Standardized, Model-Based Approach
Studies in Second Language Acquisition, v46 n1 p227-250 2024
A slowdown or a speedup in response times across experimental conditions can be taken as evidence of online deployment of knowledge. However, response-time difference measures are rarely evaluated on their reliability, and there is no standard practice to estimate it. In this article, we used three open data sets to explore an approach to reliability that is based on mixed-effects modeling and to examine model criticism as an outlier treatment strategy. The results suggest that the model-based approach can be superior but show no clear advantage of model criticism. We followed up these results with a simulation study to identify the specific conditions in which the model-based approach has the most benefits. Researchers who cannot include a large number of items and have a moderate level of noise in their data may find this approach particularly useful. We concluded by calling for more awareness and research on the psychometric properties of measures in the field.
Descriptors: Reliability, Reaction Time, Psychometrics, Criticism, Models, Individual Differences, Simulation, Measurement, Item Analysis
Cambridge University Press. 100 Brook Hill Drive, West Nyack, NY 10994. Tel: 800-872-7423; Tel: 845-353-7500; Fax: 845-353-4141; e-mail: subscriptions_newyork@cambridge.org; Web site: https://www.cambridge.org/core/what-we-publish/journals
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A