Although there is an abundance of research on the association of school poverty (or socioeconomic status) and test score level, there is very little rigorous longitudinal evidence on the cumulative effects of exposure to differing school contexts. Drawing from methods used first in epidemiology and then in neighborhood effects research, we use population-level longitudinal data from North Carolina to estimate a structural nested mean model that permits proper adjustment for time-varying confounding. Unlike panel data studies using student fixed effects, which often report close to null findings, we find evidence of modest but significant negative effects of school poverty composition on eighth-grade reading and math test scores in models that control for third-grade test scores and baseline treatment status.