Assessment in mathematics is assumed to provide credible and important information about what students know and can do. In this paper we focus on large scale tests and question whether mathematics assessment is essentially gender neutral. We consider aspects of test validity and discuss issues of terminology related to gender and mathematics. In particular, we highlight examples of the ways that test content, response processes, and interactions with other variables such as socio-economic status and beliefs/attitudes can distort students' results and affect the interpretation of achievement outcomes. Where appropriate, we highlight findings from our own research