ERIC Number: EJ1461166
Record Type: Journal
Publication Date: 2025-Mar
Pages: 22
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1560-4292
EISSN: EISSN-1560-4306
Available Date: 2025-01-28
Supporting Literacy Assessment in West Africa: Using State-of-the-Art Speech Models to Assess Oral Reading Fluency
Owen Henkel1; Hannah Horne-Robinson2; Libby Hills3; Bill Roberts4; Josh McGrane5
International Journal of Artificial Intelligence in Education, v35 n1 p282-303 2025
This paper reports on a set of three recent experiments utilizing large-scale speech models to assess the oral reading fluency (ORF) of students in Ghana. While ORF is a well-established measure of foundational literacy, assessing it typically requires one-on-one sessions between a student and a trained rater, a process that is time-consuming and costly. Automating the assessment of ORF could support better literacy instruction, particularly in education contexts where formative assessment is uncommon due to large class sizes and limited resources. This research is among the first to examine the use of the most recent versions of large-scale speech models for ORF assessment in the Global South. We find that the best performing model, Whisper V2, with no additional fine-tuning, produces transcriptions of Ghanaian students reading aloud with a Word Error Rate of 10.3. When these transcriptions are used to produce fully automated ORF scores, they closely align with scores generated by expert human raters, with a correlation coefficient of 0.98. These results were achieved on a representative dataset (i.e., students with regional accents, recordings taken in actual classrooms), using a free and publicly available speech with no additional fine-tuning. This model's strong performance on real-world classroom data, combined with its accessibility and simplified implementation, suggests potential for scaling ORF assessment in lower-resource, linguistically diverse educational contexts.
Descriptors: Foreign Countries, Oral Reading, Reading Fluency, Literacy, Error Patterns, Scores, Automation, Reading Tests, Educational Assessment, Auditory Perception
Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://bibliotheek.ehb.be:2123/
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Ghana
Grant or Contract Numbers: N/A
Author Affiliations: 1University of Oxford, Oxford, UK; 2Rising Academies, Accra, Ghana; 3Jacobs Foundation, Zurich, Switzerland; 4Legible Labs, New York, USA; 5University Melbourne, Parkville, Australia