ERIC Number: ED658586
Record Type: Non-Journal
Publication Date: 2022-Sep-21
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Widening or Closing the Gap? Examining Heterogenous Effects of Developmental Education for College for Students with Learning and Attention Disabilities
Ryan Wells; Ling Chen
Society for Research on Educational Effectiveness
Background: Developmental education (i.e., remedial education) has come under scrutiny given evidence that it is ineffective for learning or hinders students' progress toward degree completion (Scott-Clayton & Rodriguez, 2014; Xu & Dadgar, 2018). However, there is heterogeneity in the effect of developmental education, with positive effects for those who are least prepared academically (Boatman & Long, 2018). Given such findings, more attention must be given to how developmental education impacts students with a learning disability (LD) and/or attention deficit (hyperactivity) disorder (ADHD). This population often experiences systems and structures that are not suited to their needs resulting in lower rates of college readiness (Cortellia & Horowitz, 2014; Koch et al., 2018; Shifrer, 2016; Shifrer et al., 2013). Therefore, this underserved community deserves more empirical attention to understand what is effective for their learning, in alignment with the conference theme. Purpose & Research Questions: Our purpose was to estimate the effect of developmental education on a) college persistence and b) accumulation of course credits, for students with learning and attention disabilities. Specifically, we asked: (1) To what extent do the effects of participation in developmental education differ for students with LD and/or ADHD?; and (2) Within the subpopulation of students with LD and/or ADHD, to what extent do the effects of participation in developmental education differ by attendance at a two- or four-year institution? Setting & Population: The setting was U.S. postsecondary education and the population under consideration was high school graduates in 2012, who went on to attend college. The dataset used -- High School Longitudinal Study (HSLS) -- is nationally representative of this population (detailed below). Intervention: Developmental education as an intervention has positively affected persistence and completion in some cases (Bettinger & Long, 2009) and may be more effective in two-year colleges than at four-year institutions (Attewell et al., 2006). While those who are least prepared may benefit the most, longer time frames to completion may still negatively affect progress (Xu & Dadgar, 2018). Students with LD and/or ADHD are more likely to be placed into developmental education because of high schools which did not adequately prepare them for college. Despite this, there is very little research on the topic and none using cause-and-effect methods. Research Design: We used a doubly robust strategy (Ho et al., 2007) combining coarsened exact matching (CEM) with regression to estimate the effect of developmental education (see Figure 1, causal graphical model). CEM (Iacus et al., 2011) not only balances the data more effectively than propensity score and other matching techniques, but also reduces model dependence, estimation error, and bias to a greater extent (Iacus et al., 2012; King et al., 2011). CEM uses the same algorithm as traditional exact matching to find matches and trim the data. However, the researcher recodes relevant variables so that similar cases are grouped together (i.e., coarsening). Once the match takes place, un-coarsened values are used for the analyses and estimation of causal effects, such that the assumptions involved are minimal (Iacus et al., 2012). (Sensitivity analyses were also performed using inverse propensity weighting.) Data Collection & Analysis: We utilized transcript data from HSLS, which collected data from 9 graders in 2009 and followed them through 2021, when transcript data were released. These are the best available national data that include information about developmental education and postsecondary outcomes, as well as pre-treatment variables necessary to address selection bias via our analytical strategy. The sample consisted of 8932 students--1736 with LD and/or ADHD--who enrolled in a postsecondary institution. After data were matched with CEM, we used logistic regression (for the binary outcome of persistence) and linear regression (for credits earned), including covariates to adjust for confounding causes of persistence (corresponding to X in Figure 1). To examine moderation of the effect of developmental education by LD/ADHD status, we stratified regression models by LD/ADHD status. Findings: Table 1 shows descriptive analysis of the analytical sample used, following multiple imputation of missing data. For example, 63% of students in developmental education persisted, compared to 73% of other students. Table 2 shows a summary of the effects of participation, based on regression results of matched data. Students with LD and/or ADHD who participated had a probability of persisting that was three percentage points lower than students with LD/ADHD who did not participate (62% vs 65% probability). In contrast, students without LD/ADHD who were in the developmental education intervention were six percentage points higher in their probability of persisting (68% versus 62%) than similar students who did not participate. Figure 3 shows these predicted probabilities graphically. (See appendix for sensitivity analyses of all results using inverse propensity weighting, including a similar graph.) While not reaching typical levels of statistical significance, the change in opposite directions (i.e., widened gap) suggest a possible heterogeneous effect based on disability status. Table 2 shows that students with and without LD/ADHD in developmental education ended up with more credits accumulated, which could lead to longer times to degree, though the evidence is not strong. There is no evidence of a heterogeneous effect for this outcome. Addressing the second research question concerning effects within the subpopulation of students with LD/ADHD, Table 3 shows descriptive results only for that group. Table 4 shows a summary of the intervention effects, separately by two- and four-year institution. While more students with disabilities attend two-year institutions and over half of students at these institutions participate in developmental education (see Table 1), effects of the intervention do not appear to differ by institution type. Conclusions: Findings from this line of research can inform policymakers and practitioners about being more purposeful when designing developmental education reforms to best support students with learning and attention disabilities. Our results suggest that there may be heterogeneous effects of this intervention on persistence, perhaps widening already troublesome gaps in postsecondary outcomes, though additional research is needed. Given a very small literature based for this underserved group of students, this small but generative step forward in is valuable.
Descriptors: Remedial Instruction, Students with Disabilities, Learning Disabilities, Attention Deficit Hyperactivity Disorder, Program Effectiveness, College Students, Academic Persistence, College Credits, Two Year Colleges, College Environment, Probability
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: Higher Education; Postsecondary Education; Two Year Colleges
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Assessments and Surveys: High School Longitudinal Study of 2009 (NCES)
Grant or Contract Numbers: N/A