There is a problematic but widespread perception that we do mathematics with our minds, and sport and Physical Education (PE) with our bodies. Strong identification with sport and PE could therefore pose a barrier to mathematics learning. I explore this troubling possibility using data from a mixed-methods study of 450 Year Nine students (age 13-14) in three New Zealand schools. The most socially privileged students often described mathematical and sporting success as entirely compatible, although most described mathematics as valuable for their working futures and sport as enjoyable but inconsequential. In contrast, working-class and ethnic minority boys often identified strongly with 'physical' masculinity and viewed physical strength and sporting prowess as pathways to paid work. Such students tended not to identify as mathematically capable. This mechanism for exclusion from mathematical success is particularly concerning in nations such as New Zealand where sport is so dominant in school-age boys' imagined futures.