ERIC Number: EJ1310382
Record Type: Journal
Publication Date: 2021-Oct
Pages: 20
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-1756-1108
EISSN: N/A
Development of a Machine Learning-Based Tool to Evaluate Correct Lewis Acid-Base Model Use in Written Responses to Open-Ended Formative Assessment Items
Yik, Brandon J.; Dood, Amber J.; Cruz-Ramirez de Arellano, Daniel; Fields, Kimberly B.; Raker, Jeffrey R.
Chemistry Education Research and Practice, v22 n4 p866-885 Oct 2021
Acid-base chemistry is a key reaction motif taught in postsecondary organic chemistry courses. More specifically, concepts from the Lewis acid-base model are broadly applicable to understanding mechanistic ideas such as electron density, nucleophilicity, and electrophilicity; thus, the Lewis model is fundamental to explaining an array of reaction mechanisms taught in organic chemistry. Herein, we report the development of a generalized predictive model using machine learning techniques to assess students' written responses for the correct use of the Lewis acid-base model for a variety (N = 26) of open-ended formative assessment items. These items follow a general framework of prompts that ask: why a compound can act as (i) an acid, (ii) a base, or (iii) both an acid and a base (i.e., amphoteric)? Or, what is happening and why for aqueous proton-transfer reactions and reactions that can only be explained using the Lewis model. Our predictive scoring model was constructed from a large collection of responses (N = 8520) using a machine learning technique, i.e., support vector machine, and subsequently evaluated using a variety of validation procedures resulting in overall 84.5-88.9% accuracies. The predictive model underwent further scrutiny with a set of responses (N = 2162) from different prompts not used in model construction along with a new prompt type: non-aqueous proton-transfer reactions. Model validation with these data achieved 92.7% accuracy. Our results suggest that machine learning techniques can be used to construct generalized predictive models for the evaluation of acid-base reaction mechanisms and their properties. Links to open-access files are provided that allow instructors to conduct their own analyses on written, open-ended formative assessment items to evaluate correct Lewis model use.
Descriptors: Artificial Intelligence, Models, Formative Evaluation, Organic Chemistry, College Science, Student Evaluation, Accuracy, Responses, Scoring, Predictive Validity, College Students
Royal Society of Chemistry. Thomas Graham House, Science Park, Milton Road, Cambridge, CB4 0WF, UK. Tel: +44-1223 420066; Fax: +44-1223 423623; e-mail: cerp@rsc.org; Web site: http://www.rsc.org/cerp
Publication Type: Journal Articles; Reports - Research; Tests/Questionnaires
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Florida
Grant or Contract Numbers: N/A