Visuo-spatial reasoning tests, such as Raven's matrices, Cattell's culture-fair test, or various subtests of the Wechsler scales, are frequently used to estimate intelligence scores in the context of inter-racial comparisons. This has led to several high-profile works claiming that certain ethnic groups have lower intelligence than others, presumably due to genetic inferiority. This logic is predicated on the assumption that such visuo-spatial tests, because they are non-verbal, must be culture-fair: that their solution process does not significantly draw on factors that vary from one culture to the next. This assumption of culture-fairness is dubious at best and has been questioned by many authors. In this article, I review the substantial body of psychological and ethnographic literature which has demonstrated that the perception, manipulation and conceptualization of visuo-spatial information differs significantly across cultures, in a way that is relevant to intelligence tests. I then outline a model of how these inter-cultural differences can affect seven major steps of the solution process for Raven's matrices, with a brief discussion of other visuo-spatial reasoning tests. Overall, a number of cultural assumptions appear to be deeply ingrained in all visuo-spatial reasoning tests, to the extent that it disqualifies the view of such tests as intrinsically culture-fair and makes it impossible to draw clear-cut conclusions from average score differences between ethnic groups.