Language is often depicted as the sine qua non of mathematical thinking, a view buttressed by findings of language-of-training effects among bilinguals. These findings, however, have been limited to studies of arithmetic. Nothing is known about the potential influence of language on the ability to learn rules about the relations among variables (e.g., algebra). To test whether arithmetic and algebraic thinking differ, Chinese-English bilinguals were trained to solve arithmetic and algebra problems in either Chinese or English and then tested on new and old problems in both languages. For arithmetic problems, solution times were always longer for English than Chinese; in both languages, solution times dropped during training; after training, solution times continued to drop for old problems, but returned to pre-training levels for new problems. In contrast, for algebra problems, solution times did not differ across language; solution times dropped during training; after training, gains in speed were preserved for both old and new problems. These findings suggest that the contribution of language to mathematical thinking may be limited to the areas of mathematics that are learned by rote and not by rule. [This paper was published in: "Proceedings of the 40th Annual Conference of the Cognitive Science Society," edited by T. T. Rogers et al., Cognitive Science Society, 2018, pp. 918-923.]