ERIC Number: ED659343
Record Type: Non-Journal
Publication Date: 2023-Sep-30
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
A Multivariate Meta-Analysis of the Effects of Business-as-Usual Special Education Services in U.S. Schools
Adrienne D. Woods; Cecelia Gloski; Yangyang Wang; Paul Morgan
Society for Research on Educational Effectiveness
Background: Special education services are individualized supports provided to students with disabilities (SWD) that should help them make appropriate educational progress given their abilities, educational needs, and goals (Endrew F. v. Douglas County School District, 2017). Concerns about the benefits of these services have emerged given the increasing numbers of SWD and costs of providing services (Chambers et al., 2014; NCES, 2022), the widening opportunity gap between students with and without disabilities (NCES, 2021; Gilmour et al., 2013), the potential for stigmatization that may result from being labeled as needing services (Shifrer, 2013), and observations that special education services are often insufficiently intensive to result in appropriate educational progress (Gilmour et al., 2013; Vaughn & Wanzek, 2014). Yet, conclusions about the effects of being assigned to special education that are derived from correlational research or from studies that do not adjust for potential confounds (including adjusting for baseline "prima facie" differences between students with and without disabilities) can lead to mischaracterizations of their effectiveness. Objectives: To our knowledge, no review has assessed the state of causal evidence for the effectiveness of special education services. We help address this gap by examining special education treatment effects, both overall and across several related outcome domains, in a moderated, multivariate, multilevel meta-analysis framework. Results offer the field an evidence-based baseline of the extent to which receiving U.S. special education services may affect common academic and non-academic learning outcomes. We also evaluated whether these effects may be moderated by two different adjustments for baseline differences between students: whether the study used a quasi-experimental design [QED] or descriptive design, and whether effects were compared among SWD who vary in the amount and timing of their special education services, or relative to students who receive general education. Research Design: We searched several scholarly search engines and databases for studies meeting the following criteria: (1) Examined the relation between receiving business-as-usual special education services (i.e., not specific interventions or programs) and academic, behavioral, and/or social-emotional outcomes; (2) Article was peer-reviewed or grey literature (e.g., dissertation, working paper); (3) Occurred in pre-K through 12th grade settings in the United States; (4) Reported at least some numerical results corresponding to the effects of receiving special education; and (5) Utilized a QED or a descriptive research design (controlled for at least one covariate). This led to a final sample size of K = 40 studies using data from 23 datasets and N = 1,721 effect sizes for inclusion in our meta-analysis. Our full search process is displayed in Figure 1. Population: Sample sizes ranged from n = 68 to n = 14.4 million students (median treatment n = 690; median control n = 1,175). The mean age at special education service receipt was 9.09 years, and the mean age at outcome measurement was 10.62 years. See Table 1 for additional study information. Analysis: We analyzed multivariate effect sizes in a random-effects meta-analysis to deal with the potential for both nonindependence and correlated outcomes (Assink & Wibelink, 2016; Cheung, 2015; Konstantopoulos, 2011). We evaluated random rather than fixed effects because the underlying "true" effect sizes may differ across studies (Harrar et al., 2021). Because we often extracted multiple effect sizes from the same study (e.g., both achievement and behavior outcomes), and because some studies used the same datasets, we evaluated a threelevel meta-analysis in which participants were nested within studies, who were nested within dataset (Colich et al., 2020). We also used meta-regression to explore potential moderators of effects including type of outcome and study design/comparison group. All analyses were conducted in R version 4.1.1. Findings/Results: Results are presented in Table 2. As study designs more rigorously adjust for baseline differences (e.g., from left to right; columns indicate all included studies, QEDs, studies that only included SWD, and QEDs that only included SWD), more positive effects of receiving special education services are noted, suggesting that descriptive studies contrasting students with and without disabilities may be biased by baseline differences even with covariate adjustment. There were larger, positive effects for graduation (z = 0.16-0.24), postsecondary outcomes (z = 0.15-0.23), and other outcomes (z = 0.26-0.38) across study design. The average effect of special education on achievement reversed in directionality across study design, increasing from negative but non-significant (z = -0.07) to both positive and significant among QEDs that only included contrasts among SWD (z = 0.12). Similarly, the negative effect noted for behavior across all studies diminished in magnitude to an effect size that was not meaningfully or significantly different from zero among these more rigorous study designs (z = -0.07 to -0.01). We found little evidence for other moderation effects (Table 3) and no evidence of publication bias. Conclusions: We examined the effects of assignment to business-as-usual special education services rather than specific interventions or programs. We were unable to examine for the exact special education services or evidence-based practices delivered to SWD. The impact of special education services on academic achievement was positive but small in effect size across study designs that made a better attempt to adjust for baseline differences, ranging from z = 0.04 to 0.12. This result suggests that special education services may not be sufficiently intensive to improve academic achievement. Special education services may have a negative effect at worst (z = -0.13) and a neutral effect at best (z = -0.01) on student behavior. This finding is consistent with the idea that special education services might have both positive and negative effects on SWD. However, and optimistically, our results also imply that despite any immediate negative repercussions, referral for special education services may confer long-run benefits to SWD. The effects of special education may be especially positively associated with non-academic outcomes (e.g., truancy, grade repetition, physical health, college preparation, or family/home outcomes), with effect sizes ranging from z = 0.21 to 0.38. The relative lack of studies that utilize QEDs comparing SWD to other SWDs suggests that further causal research is needed to understand the mechanisms by which special education may bestow this longitudinal advantage.
Descriptors: Meta Analysis, Special Education, Students with Disabilities, Access to Education, Educational Opportunities, Correlation, Referral, Evidence Based Practice, Outcomes of Education, Research Design, Search Engines, Databases, Research Reports, Kindergarten, Elementary Secondary Education, Effect Size, Comparative Analysis, Academic Achievement, Quasiexperimental Design
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: Early Childhood Education; Elementary Education; Kindergarten; Primary Education; Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Grant or Contract Numbers: N/A