ERIC Number: ED593103
Record Type: Non-Journal
Publication Date: 2018-Jul
Pages: 11
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
An Open Vocabulary Approach for Estimating Teacher Use of Authentic Questions in Classroom Discourse
Cook, Connor; Olney, Andrew M.; Kelly, Sean; D'Mello, Sidney K.
International Educational Data Mining Society, Paper presented at the International Conference on Educational Data Mining (EDM) (11th, Raleigh, NC, Jul 16-20, 2018)
Automatic assessment of the quality of classroom discourse can have a transformative effect on research and practice on improving teaching effectiveness. We improve on a previous automated method to measure teacher authentic questions -- open-ended questions without pre-scripted responses that predict student achievement growth -- using classroom audio and expert question codes from two sources: (1) a large archival database of text transcripts of 428 class-sessions from 116 classrooms, and (2) a newly collected sample of 132 high-quality audio recordings with automatic speech recognition transcripts from 27 classrooms. Whereas previous work utilized a "closed vocabulary" approach, consisting of 732 pre-defined word, sentence, and discourse level features, the present "open vocabulary" approach exclusively utilized word and phrase counts from the transcripts themselves. The two approaches yielded substantial, but statistically equivalent, correlations with gold-standard human codes of authenticity (Pearson r's of 0.396 vs. 0.424 and 0.602 vs. 0.613 for datasets 1 and 2, respectively). Importantly, averaging estimates from the two approaches resulted in statistically significant improvements over either approach (r's of 0.492 and 0.686 for datasets 1 and 2, respectively). We discuss implications of our findings for automated analysis of classroom discourse. [For the full proceedings, see ED593090.]
Descriptors: Vocabulary, Academic Discourse, Teacher Effectiveness, Partnerships in Education, Observation, Teaching Methods, Correlation, English Instruction, Language Arts, English Teachers, Grade 7, Grade 8, Middle School Students, Middle School Teachers, Questioning Techniques
International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: http://www.educationaldatamining.org
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: Grade 7; Junior High Schools; Middle Schools; Elementary Education; Secondary Education; Grade 8
Audience: N/A
Language: English
Sponsor: National Science Foundation (NSF); Institute of Education Sciences (ED)
Authoring Institution: N/A
Identifiers - Location: Wisconsin; New York
IES Funded: Yes
Grant or Contract Numbers: NSFIIS1735785; R305A130030