The South Australian Certificate of Education (SACE) is a credential and formal qualification within the Australian Qualifications Framework. A recent review of the SACE outlined a number of recommendations for significant changes to this certificate. These recommendations were the result of a process that began with the review panel "scrutinizing carefully [existing SACE structures for] continuing validity and effectiveness". This paper critiques the "careful examination" of statistical trends and patterns used to build the case for reform. Central to these trends and patterns are measures of retention, socio-economic status and student achievement, all of which are problematic. This paper also challenges the appropriateness of the statistical techniques used in the review. The paper concludes by arguing that making significant policy changes based upon such limited and flawed analyses is problematic. (Contains 6 tables and 5 figures.)