ERIC Number: ED659758
Record Type: Non-Journal
Publication Date: 2023-Sep-28
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Understanding the Heterogeneous Impacts of Conditional Cash Transfer Programs on Academic Participation
JoonHo Lee
Society for Research on Educational Effectiveness
Background/Context: Conditional cash transfers (CCTs) have become one of the most prevalent and rapidly growing social assistance strategies in developing countries. Typically, education-focused CCT programs provide monthly or bimonthly cash payments to students who meet specific attendance requirements. There is a wealth of literature documenting the positive effects of CCTs on outcomes such as enrollment, attendance, and dropout rates across various countries and contexts (see Garcia & Saavedra, 2017 for a comprehensive review). CCT interventions are often implemented across multiple school sites (e.g., Barrera-Osorio et al., 2011; 2019). While understanding the overall impact of a CCT program is essential for informing policy recommendations, it is also crucial to investigate questions related to cross-site variation in CCT effects in order to gain a deeper understanding of how CCTs function and under which circumstances they are most or least effective. In the present study, we employ rigorous analytical techniques to quantify and characterize the heterogeneity of CCT program impacts across different sites. As a case example, we utilize data from the Conditional Subsidies for School Attendance (Subsidios Condicionados a la Asistencia Escolar) program implemented in Bogota, Colombia. This experimental evaluation of a CCT intervention aimed to improve student retention, reduce dropout rates, and decrease child labor among low-income secondary school students (Barrera-Osorio et al., 2019). Purpose/Objective/Research Question: This study addresses three research questions: 1. What is the overall impact of CCT programs on secondary education outcomes, including on-time enrollment, dropout rates, and secondary graduation, as well as medium-term tertiary education enrollment in universities, vocational institutions, and unclassified establishments? 2. To what extent do the average treatment effects (ATEs) of CCT programs vary across school sites in terms of both secondary and tertiary education outcomes? 3. How much of the variation in CCT program effects across school sites can be attributed to "compositional" differences in student populations, such as family income and parents' education levels, as opposed to "contextual" differences in school-level characteristics? If all sites had the same population of students, what would be the ATEs for each site? Setting/Population/Intervention/Design: In 2005, the Secretary of Education in Bogota, Colombia's capital city, implemented the CCT program in two of the poorest districts among the 20: San Cristobal and Suba. In the San Cristobal district, students in grades 6 to 11 were randomly assigned to one of three CCT experimental groups: "basic" treatment, "savings" treatment, or control (refer to Barrera-Osorio et al., 2019 for details). In the Suba district, participants in grades 9 to 11 were randomly assigned to one of two groups: "tertiary" treatment or control. These two multisite experiments involved a total of 13,491 participants across 99 sites within the two districts (see Figures 1 and 2). Barrera-Osorio and colleagues combined the original program registration and randomization records with administrative data sources to track educational outcomes over time. As primary outcomes, we analyze on-time secondary school enrollment, secondary school exit exam participation, and tertiary enrollment by institution type. Data Analysis: To address Research Questions 1 and 2, the main challenge involves utilizing the observed heterogeneity in estimated effects (see Figures 3 and 4) as an indicator of heterogeneity in effects across a broader range of sites, while accounting for the sampling variation that contributes to some of the observed heterogeneity (Rubin, 1981; Meager, 2019). We employ the Bayesian hierarchical framework to distinguish genuine heterogeneity from sampling variation (refer to the Appendix for details). To quantify the heterogeneity of site-specific treatment effects of CCT programs, we calculate and report the shrinkage factors for each outcome, which measure the proportion of total variation attributable to between-site variation relative to sampling variation. To address Research Question 3, we apply the reweighting method developed by Lu et al. (2021) to decompose treatment effect variation resulting from "compositional" differences in the distributions of student-level covariates from variation due to "contextual" differences in school-level features. Their method re-weights (or "transports") each school to have a common distribution of student-level covariates, allowing the remaining variation in impact to capture contextual differences between sites. We compute the approximate balancing weights using the balancer R package. Findings/Conclusions: "The general impact of CCTs on educational outcomes." Our findings show that the ATEs on secondary and tertiary education outcomes generally align with the patterns reported by Barrera-Osorio et al. (2019). However, when considering cross-site impact variations, the estimated ATEs reveal an increased posterior probability of a null intervention effect due to uncertainty at both within-site and between-site levels (see Figures 5-8). The fixed-effects approach, which does not account for variation between sites, may lead to an underestimation of the predicted CCT effect's uncertainty in new or future contexts. "Treatment effect heterogeneity of CCT programs." The estimated shrinkage factor metrics range from 0.02 to 0.29, with the majority of values falling between 0.05 and 0.12. This indicates that approximately 88% to 95% of the initially observed variation in CCT treatment effects is due to within-site sampling variation (see Table 1). The cross-site impact variations, measured in effect size units, are estimated to range from 0.04 to 0.22, with the 25th and 75th percentiles being 0.07 and 0.14, respectively. According to the definition provided by Weiss et al. (2017), the cross-site impact variations observed in the CCT program are considered modest or moderate. "Decomposition of compositional and contextual effects." The reweighting did not significantly alter the distribution of site-specific ATEs; however, some site-specific ATE estimates were modified. We estimate that less than 8.2% of the variance in site-specific ATEs can be explained by observed differences in student-level covariates. Thus, we conclude that variations in impact across sites in these CCT experiments are primarily driven by contextual differences in site-level characteristics.
Descriptors: Financial Support, Educational Finance, Attendance, Program Implementation, Foreign Countries, School Holding Power, Dropout Rate, Dropout Prevention, Child Labor, Developing Nations, Secondary School Students, Program Effectiveness, Family Income, Parent Background
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Location: Colombia (Bogota)
Grant or Contract Numbers: N/A