Under the Individuals with Disabilities Education Act, schools are required to provide a free appropriate public education to students with disabilities and show that the students are making academic progress. This study compares within- and across-years academic growth from kindergarten to 4th grade for students who were ever in special education (ever-SPED) to students who were never in special education (never-SPED). We use math and reading assessment scores from NWEA's Growth Research Database. Our dataset contains one cohort of about 4,200 students, assessed up to three times per year for five years. Although ever-SPED students started kindergarten with lower math and reading test scores and grew less in both subjects than never-SPED students during the kindergarten school year, ever-SPED students grew more than never-SPED students during the 1st and 2nd grade school years in math and 1st, 3rd, and 4th grade school years in reading. However, ever-SPED students lost more learning during every summer than never-SPED students. This led the test score disparities between the two to grow from under 0.5 standard deviations in kindergarten to 1.0 standard deviation in 4th grade. These findings suggest that summer learning opportunities are crucial for improving educational outcomes for students with disabilities.