ERIC Number: EJ1420873
Record Type: Journal
Publication Date: 2024
Pages: 16
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: EISSN-2469-9896
Performance of ChatGPT on the Test of Understanding Graphs in Kinematics
Physical Review Physics Education Research, v20 n1 Article 010109 2024
The well-known artificial intelligence-based chatbot ChatGPT-4 has become able to process image data as input in October 2023. We investigated its performance on the test of understanding graphs in kinematics to inform the physics education community of the current potential of using ChatGPT in the education process, particularly on tasks that involve graphical interpretation. We found that ChatGPT, on average, performed similarly to students taking a high-school level physics course, but with important differences in the distribution of the correctness of its responses, as well as in terms of the displayed "reasoning" and "visual" abilities. While ChatGPT was very successful at proposing productive strategies for solving the tasks on the test and expressed correct reasoning in most of its responses, it had difficulties correctly "seeing" graphs. We suggest that, based on its performance, caution and a critical approach are needed if one intends to use it in the role of a tutor, a model of a student, or a tool for assisting vision-impaired persons in the context of kinematics graphs.
Descriptors: Computer Software, Artificial Intelligence, Visual Impairments, Graphs, Physics, Problem Solving, Teaching Methods, Technology Uses in Education, Motion, Mechanics (Physics)
American Physical Society. One Physics Ellipse 4th Floor, College Park, MD 20740-3844. Tel: 301-209-3200; Fax: 301-209-0865; e-mail: assocpub@aps.org; Web site: https://journals.aps.org/prper/
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A