NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: ED663007
Record Type: Non-Journal
Publication Date: 2024-Sep-19
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Comparing Approaches to Predicting Risk for Dyslexia
Patrick Kennedy; Brian Gearin; Katherine Bromley; Gina Biancarosa
Society for Research on Educational Effectiveness
Background: Dyslexia is a specific reading disability characterized by word recognition difficulties that can qualify a student for special education services under the "Individuals with Disabilities Education Act" (IDEA; 2004; Yudin, 2015). As of 2024, 49 US states have legislation defining dyslexia and enumerating for schools recommended or required dyslexia practices (National Center on Improving Literacy [NCIL], 2024). This legislation drives widespread universal screening and intervention efforts, as 39 states require that students be screened for dyslexia, 31 of which also require intervention based on those screening results (NCIL, 2024). Further, although schools are typically mandated to use screeners that assess foundational early literacy constructs, to date, no assessments have been specifically validated for the purpose of screening for dyslexia. Purpose: This study investigates the accuracy, validity, utility, and efficiency of various approaches to screening for dyslexia, using multiple longitudinal administrations of DIBELS 8 (University of Oregon, 2018), a literacy screener that is both widely used in schools and widely evoked in state dyslexia policy. Given substantial variation in state dyslexia definitions, we operationalize dyslexia using multiple measurement schemes, prioritizing the hybrid model of dyslexia described in Miciak and Fletcher (2020) and illustrated in Figure 1. Setting: In 2021-2022 and 2022-2023, 36 schools from 20 states representing all four US Census regions administered DIBELS 8 and standardized measures of word reading, phonological awareness, spelling, and vocabulary to an ability-stratified sample of Grade K-3 students. Prior to recruitment, schools were categorized into one of five strata based on The Generalizer, a tool for designing educational evaluations with a population perspective (Tipton, & Miller, 2024). See Figures 2 and 3 for a summary of the states represented and the school-level demographics used to stratify the sample. Participants: Across two years, measures were administered to 2,834 Grade 3 students, 2,342 Grade 2 students, 1,728 Grade 1 students, and 960 kindergarten students. Twenty-eight schools participated in both years, resulting in up to six DIBELS administrations and two sets of outcome measures on approximately 75% of participating students to date. Research Design: Each cohort of students is being followed through the end of third grade, or the end of the 2024-2025 school year, whichever comes first. The population of interest is students with or at risk for dyslexia and who are representative of similar students across the country. Thus, although schools typically administer DIBELS 8 to all students in participating grades, to reduce the testing burden, students selected for assessment with the standardized measures were sampled from across the range of ability levels. Students were grouped into quartiles based on their beginning of year (BOY) DIBELS 8 composite score and project staff randomly selected seven students from each of the two lowest quartiles, six students from the third quartile, and five students from the top quartile, resulting in a sample with varying proficiency levels. Data Collection and Analysis: Participating students were assessed three times per year on DIBELS 8 measures of phonological awareness, alphabetic principle, word knowledge, accuracy and fluency with connected text, and comprehension, and once at the end of the year (EOY) on a standardized measure of word reading (Torgesen et al., 2012). Using a planned missingness design (Little & Rhemtulla, 2013), students in each school additionally received either one or all three of the other standardized assessments: an individually administered reading readiness and reading achievement measure (Woodcock, 2011); a norm-referenced, group administered test of spelling (Larsen, Hammill, & Moats, 2013); and an individually administered, norm-referenced vocabulary assessment (Martin & Brownwell, 2011). See Table 1 for additional details regarding the alignment of measures to relevant hybrid model constructs. Variations on three analytic approaches are compared: ROC curve analyses (Habibzadeh, Habibzadeh, & Yadollahie, 2016), conceptual replications of a hybrid model approach that uses multiple sources of information to define reading disability (Schatschneider et al., 2016), and machine learning (ML) decision tree models that predict the hybrid model of dyslexia. Results: Using traditional ROC curve approaches, DIBELS 8 is highly predictive of individual standardized assessments in all grades. As shown in Figure 4, DIBELS 8 measures of Nonsense Word Fluency (NWF), Word Reading Fluency (WRF), Oral Reading Fluency (ORF), and the Composite score at all three times of year are highly predictive of the end of year (EOY) TOWRE Word Reading Efficiency Composite in grades 1 and 2, with all AUC values exceeding 0.80, and 20 of the 30 possible comparisons exceeding 0.90. Schatschneider et al. (2016) represents one of the only investigations to date into reading difficulty incidence rates and longitudinal classification stability but differs from these analyses in that results are based on simulated data. We found important and unexpected differences in incidence rates relative to those reported in Schatschneider et al., which are likely to impact longitudinal stability estimates. Specifically, present incidence rates suggest that the longitudinal stability of hybrid models are likely to be higher if the cut used is the 25th percentile, but slightly lower if the 15th percentile is used, and substantially lower if the 5th percentile is used. From a ML perspective, we compare the effectiveness of two decision tree-based methodologies: Classification and Regression Trees and Random Forest analyses using initial status and slopes for DIBELS 8 measures as predictors. Analyses are ongoing, but given findings to date, we expect these ML approaches that incorporate multiple measures and administrations to improve our ability to predict performance on a latent dyslexia construct, but that the stability of those estimates may differ depending on how dyslexia is operationalized, and the cut points chosen. Conclusions: Findings from this study offer educational stakeholders valuable information regarding the adequacy of using DIBELS 8 to screen for dyslexia longitudinally. Results to date suggest that, from the perspectives of content coverage and psychometric evidence, DIBELS 8 is well positioned to serve as a screener for dyslexia. Important questions remain regarding how to best operationalize risk for dyslexia and identify cut scores that balance accuracy, validity, and efficiency concerns.
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: Elementary Education; Early Childhood Education; Kindergarten; Primary Education; Grade 1; Grade 2; Grade 3
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Laws, Policies, & Programs: Individuals with Disabilities Education Improvement Act 2004
Grant or Contract Numbers: N/A