NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: ED658617
Record Type: Non-Journal
Publication Date: 2022-Sep-24
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Understanding High Schools' Effects on Longer-Term Outcomes
Preeya Mbekeani; John Papay; Ann Mantil; Richard Murnane
Society for Research on Educational Effectiveness
Background/Context: Parents and policymakers have long believed that some schools are more effective than others. State testing requirements under No Child Left Behind helped publicize the widely understood fact that schools differ substantially in their students average test scores and treated such scores as key proxies for school quality. More recent research and policy activity has focused on using test-score growth to help account for the sorting of students to schools. Raising test scores, however, is only one goal of schooling. In this study, we focus on how high schools affect students longer-run outcomes, including educational attainments and labor market earnings at age 30. While there have been recent efforts to explore how high schools promote non-test-score outcomes, such as social-emotional learning and academic engagement (e.g., Jackson et al., 2020), such outcomes are short-term in nature and proxies for what many parents, educators, and policymakers really want to know whether schools promote students development of the skills, knowledge, and attitudes that enable them to succeed in college and career. There is much less evidence about how schools affect students longer-run educational and labor-market outcomes and whether the schools in which students show more growth in short-term outcomes are those that also promote students longer-run outcomes. We posit a multidimensional framework of school effectiveness and examine whether high schools that improve particular short-term outcomes more than expected also have larger than expected effects on longer-run outcomes. Research Questions: In this study, we examine two questions--(1) Are some high schools more effective than others in promoting students four-year college graduation and adult earnings?; and (2) Are schools with larger than expected effects on these longer-run outcomes also those that improve test scores, improve students social skills, and/or promote college going? Setting & Sample: The setting for this study is Massachusetts. Our sample comprises students who entered 9th grade in a Massachusetts public high school during the 2002-03 and 2003-04 school years, providing a long enough time horizon to examine labor market earnings. We focus our analyses on 52,066 students enrolled in 105 public high schools that serve large numbers of low-income students. Data: We address our questions using detailed student-level longitudinal data from Massachusetts that include information on students longer-run educational attainments and earnings. Beyond the measures typically included in administrative data, we also have information on parent education, school climate, student behavior, and course taking from student questionnaires. Research Design: Our central question predictor is the first high school a student attended. We estimate school value-added models, conditioning on lagged outcomes and a rich set of additional covariates, to identify high schools causal impacts on four-year college completion and earnings at age 30. Past research suggests that such models yield results comparable to those from lottery-based studies (Deming, 2014). Specifically, we fit two-level mixed models with random effects for high schools. These models are of the general form: Y_ijt=?+?X_i+??S?_i+?_t+(u_j+?_it) Y_ijt represents one of our longer-term outcomes for student (i) in school (j) and cohort (t). The standard control variables are represented by the vector X_i and the expanded controls from the student survey are represented by the vector S_i. We pool data across the two cohorts and include a fixed effect for cohort, ?_t. ?_it is the random individual-level error term. We address our first question by extracting school-specific random effects, captured by u_j, and model-based estimates of the variance of school effects (???_u[caret]2) for a given student outcome. Our rich data enable us to control for a more robust set of covariates including parental education and we find strong evidence that we have accounted sufficiently for the sorting of students to schools. In particular, once we condition on this rich set of covariates, we find nearly identical estimates of school effects whether we control for family income or not, suggesting that we have resolved unobserved sorting on income, a key predictor of long-run outcomes. To address our second question, we fit models of the form described above, but with four short-term measures as the outcome. We estimate correlations between schools' value-added on these measures with their value-added on the two longer-run outcome measures. These correlations provide evidence of whether schools with larger than expected effects on longer-run outcomes are also those that improve test scores, improve students social skills, promote college going, or support students to be academically on-track. Results: We find substantial variation in high schools' effects on four-year college graduation rates and adult earnings. In Figure 1, we present predicted probabilities of four-year college graduation for a student with sample average characteristics in each high school. The probabilities range from six to 35 percentage points. In other words, for students with similar 8th grade test scores, demographic characteristics, and survey responses, there is substantial variability in longer-run outcomes based on which high school they attend. Model-based estimates of the standard deviation of high schools' estimated effects on four-year college graduation is 6.0 percentage points and just over $3,600 (in 2021 dollars) for earnings at age 30. Schools with larger estimated effects on 10th grade test scores have larger estimated effects on both four-year college graduation (r=0.487) and earnings (r=0.326). We explore proxies for schools college-going culture. We find that schools with larger impacts on students 10th grade plans to attend a four-year college also have larger effects on graduation (r=0.562) and earnings (r=0.262). Relationships between other short-term non-test score outcomes and longer-run outcomes are similarly positive but smaller in magnitude, even after disattenuating for measurement error. We present the unadjusted and adjusted correlations in Table 1. In addition to the short-term outcomes we constructed, peer effects also account for some of schools' effects, particularly in the case of earnings. Conclusions: Our results indicate that school quality matters for longer-term outcomes. While we are unable to make causal claims about particular improvement strategies schools should pursue, we demonstrate that some high schools have larger effects.
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: High Schools; Secondary Education; Higher Education; Postsecondary Education; Grade 9; Junior High Schools; Middle Schools; Elementary Education; Grade 8
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Location: Massachusetts
Grant or Contract Numbers: N/A