School inspections are a common feature of many education systems. These may be informed by quantitative background data about schools. It is recognised that there are pros and cons of using such quantitative information as part of the inspection process, though these have rarely been succinctly set out. This paper seeks to fill this gap by presenting arguments both for and against the use of quantitative data in informing school inspections. We argue that while quantitative data provide objective information about important outcomes, their usefulness is limited somewhat by a range of factors including missing data, small sample sizes, the creation of perverse incentives, and the fact that most readily available measures capture aspects other than school quality. We conclude by discussing how the Office for Standards in Education, Children's Services and Skills (Ofsted) -- the school inspectorate in England -- currently makes the trade-off between these pros and cons.