This longitudinal study examined how language ability relates to mathematical development in a linguistically and ethnically diverse sample of children from 6 to 9 years of age. Study participants were 75 native English speakers and 92 language minority learners followed from first to fourth grades. Autoregression in a structural equation modeling (SEM) framework was used to evaluate the relation between children's language ability and gains in different domains of mathematical cognition (i.e., arithmetic, data analysis/probability, algebra, and geometry). The results showed that language ability predicts gains in data analysis/probability and geometry, but not in arithmetic or algebra, after controlling for visual-spatial working memory, reading ability, and sex. The effect of language on gains in mathematical cognition did not differ between language minority learners and native English speakers. These findings suggest that language influences how children make meaning of mathematics but is not involved in complex arithmetical procedures whether presented with Arabic symbols as in arithmetic or with abstract symbols as in algebraic reasoning. The findings further indicate that early language experiences are important for later mathematical development regardless of language background, denoting the need for intensive and targeted language opportunities for language minority and native English learners to develop mathematical concepts and representations. (Contains 4 tables.)