ERIC Number: ED663634
Record Type: Non-Journal
Publication Date: 2024-Sep-20
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
What Are We Missing? Evaluating Approaches Accounting for Missing Data in Children's Word Dictation Responses
Jechun An
Society for Research on Educational Effectiveness
Teachers need instructionally useful data to make timely and appropriate decisions to meet their students with intensive needs (Filderman et al., 2019). Teachers have still experienced difficulty in instructional decision making in response to students' CBM data (Gesel et al., 2021). This is because data itself that was used for simply determining whether students require intensive needs based on level of performance may have limited direct use for teachers' instruction. The purpose of this study is to evaluate word dictation skills of elementary students who are struggling with writing using classical approach considering writing accuracy and item tree model approach considering both writing accuracy and speed. From this purpose, this study is to examine the extent to which the students' results differed by comparing the classical item response theory (IRT) approach, latent regression model (LRM) and item response tree (IRTree) model approach. For this purpose, specific research questions are presented as follows: Research Question 1. Do the IRTree and LRM approaches outperform the classical IRT when considering not-reached items? Research Question 2. Do results derived from three different approaches provide different information about students' writing abilities? Research Question 3. Do results of the first set of Word Dictation derived from different approaches could be reliably replicated with the second set of Word Dictation? This current study will utilize data from a larger research project that evaluated the effectiveness of a professional development program. This larger program was specifically designed to support elementary teachers in implementing data-based instruction for students who are struggling with writing. Participants were recruited in two sites in the Midwest in the United States of America. A total of 523 elementary students participated in the screening tests for determining eligibility for the larger program. CBM-Writing Word Dictation, used for screening tests, was designed to measure transcription skills at the word level by matching sounds and letters correctly to spell words. To perform the analysis for addressing the three research questions, Rasch dichotomous modeling will be conducted using "flirt" and "mirt" packages in R to model classical, IRTree model, and LRM approaches. The study results examined whether the results from different approaches considering not-reached items (IRTree model and LRM) may yield different ranges of writing ability level, even though students have the same score evaluated by classical IRT approach. Ultimately, results of this study will inform how instructional approaches may differ based on the potential classification by different learner types (i.e., high speed & low accuracy, high speed & high accuracy, low speed & high accuracy, and low speed & low accuracy).
Descriptors: Educational Research, Research Problems, Elementary School Students, Writing Skills, Writing Evaluation, Item Response Theory, Accuracy, Regression (Statistics), Writing Ability
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: Elementary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Grant or Contract Numbers: N/A