ERIC Number: EJ1428434
Record Type: Journal
Publication Date: 2024-Jun
Pages: 28
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1360-2357
EISSN: EISSN-1573-7608
Validating a Novel Digital Performance-Based Assessment of Data Literacy: Psychometric and Eye-Tracking Analyses
Fu Chen; Ying Cui; Alina Lutsyk-King; Yizhu Gao; Xiaoxiao Liu; Maria Cutumisu; Jacqueline P. Leighton
Education and Information Technologies, v29 n8 p9417-9444 2024
Post-secondary data literacy education is critical to students' academic and career success. However, the literature has not adequately addressed the conceptualization and assessment of data literacy for post-secondary students. In this study, we introduced a novel digital performance-based assessment for teaching and evaluating post-secondary students' data literacy skills. The purpose of this study is to validate the assessment and identify problematic items for later modifications using the argument-based approach to validation. We analyzed students' item responses and eye movements using psychometric and eye-tracking analyses to collect two types of validity evidence: internal structure and response processes. Descriptive and psychometric results showed that the nine example items measuring basic data analysis were of acceptable psychometric quality. The eye-tracking analysis of two representative items indicated that most students first attended to and processed expected item areas when the items were available. In addition, the critical item regions for task success were associated with students' highest cognitive effort. These results rejected our alternative score interpretation that the developed assessment questions evaluate students' abilities that are weakly connected to the skill of basic data analysis. Possible explanations of the findings and theoretical and pedagogical implications of our study were discussed.
Descriptors: Performance Based Assessment, College Students, Information Literacy, Evaluation Methods, Test Items, Test Validity, Responses, Eye Movements, Psychometrics
Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://bibliotheek.ehb.be:2123/
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A