The two authors conducted a small empirical study to attempt to find support for--or evidence against--the view that international tests of English language proficiency are unfair to speakers of non-standards forms of English, since these tests privilege standard forms. We explore the question of whose norms should be imposed in these tests, and what the consequences for test-takers are if the norm imposed by the test is not the "normal" variety accepted in their own society. Data used for the study are written texts by English learners from six language backgrounds, scored by raters from their own language backgrounds as well as by native American English raters. Interesting patterns emerge, but we conclude that the complexity of the variables involved, the small n-size, and the inherent unreliability of scoring productive samples prevent any definitive claims being made.