NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: EJ1434299
Record Type: Journal
Publication Date: 2024
Pages: 17
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: EISSN-1479-4403
Rubric Development and Validation for Assessing Tasks' Solving via AI Chatbots
Mohammad Hmoud; Hadeel Swaity; Eman Anjass; Eva María Aguaded-Ramírez
Electronic Journal of e-Learning, v22 n6 p1-17 2024
This research aimed to develop and validate a rubric to assess Artificial Intelligence (AI) chatbots' effectiveness in accomplishing tasks, particularly within educational contexts. Given the rapidly growing integration of AI in various sectors, including education, a systematic and robust tool for evaluating AI chatbot performance is essential. This investigation involved a rigorous process including expert involvement to ensure content validity, as well as the application of statistical tests for assessing internal consistency and reliability. Factor analysis also revealed two significant domains, "Quality of Content" and "Quality of Expression", which further enhanced the construct validity of the evaluation scale. The results from this investigation robustly affirm the reliability and validity of the developed rubric, thus marking a significant advancement in the sphere of AI chatbot performance evaluation within educational contexts. Nonetheless, the study simultaneously emphasizes the requirement for additional validation research, specifically those entailing a variety of tasks and diverse AI chatbots, to further corroborate these findings. The ramifications of this research are profound, offering both researchers and practitioners engaged in chatbot development and evaluation a comprehensive and validated framework for the assessment of chatbot performance.
Academic Conferences Limited. Curtis Farm, Kidmore End, Nr Reading, RG4 9AY, UK. Tel: +44-1189-724148; Fax: +44-1189-724691; e-mail: info@academic-conferences.org; Web site: https://academic-publishing.org/index.php/ejel/index
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A