Providing evidence that can inform awarding is an important application of Comparative Judgement (CJ) methods in high-stakes qualifications. The process of marking scripts is not changed, but CJ methods can assist in the maintenance of standards from one series to another by informing decisions about where to place grade boundaries or cut scores. The research described in this article set out to increase understanding of the risks associated with this use of CJ. Specifically, the research explored how robust the outcomes of CJ-based awarding activities would be to mismarking in the script evidence. In this article, the author explores the extent to which the simplified pairs (SP) method would be robust to mismarking in the sample of scripts used for the comparison exercise. The research has two main conclusions: (1) The SP method appears robust to single large marking errors and to fairly large marking errors in quite high proportions of sampled scripts; and (2) The SP method is more vulnerable to a general degradation of marking quality.