The question of grouping students into homogeneous and heterogeneous groups is not new, but it does not find an unambiguous answer in the literature, especially in mathematics. In this paper, we address the question of whether grouping students into homogeneous and heterogeneous groups in mathematics improves their knowledge. The quasi-experiment involved 126 8-grade (i.e., 13-14 years old) Slovenian primary school students, who were divided into two equal groups: the control group worked in homogeneous groups, and the experimental worked in heterogeneous groups. The results of the post-test show that the students from the experimental group had better results in mathematics, which indicates that heterogeneous groups should be preferred in mathematics. Lesson observations have identified differences in teacher behavior: educators working in homogeneous groups tend to give students with lower competencies tasks from lower taxonomic levels, and teachers favor abler groups. Such differences have not been observed among teachers working in heterogeneous groups. The implications for educators are also discussed.