Using kindergarten up to fourth-grade data from the Early Childhood Longitudinal Study (2010--2011 cohort), we investigated systematic variability in English language learners' (ELLs; n = 303) mathematics growth as well as relations of kindergarten language growth and working memory (WM) to ELLs' mathematics growth. Using growth mixture modeling, only one class of growth emerged from ELLs' English mathematics growth from first through fourth grades. WM related to ELLs' English mathematics growth from Grades 1 to 4, as did kindergarten growth in English early literacy. We also investigated kindergarten to Grade 4 mathematics growth between ELLs and English proficient students (EPSs; n = 4,711) using latent change score models and whether WM differentially predicted growth patterns. ELLs and EPSs did not exhibit markedly different growth patterns, and WM similarly predicted these patterns. Implications for future research as well as practical implications and limitations are discussed.