In this paper, we analyse a large, opportunistic dataset of responses (N = 219,826) to online, diagnostic multiple-choice mathematics questions, provided by 6-16-year-old UK school mathematics students (N = 7302). For each response, students were invited to indicate on a 5-point Likert-type scale how confident they were that their response was correct. Using demographic data available from the online platform, we examine the relationships between confidence and facility (the proportion of questions correct), as well as gender, age and socioeconomic disadvantage. We found a positive correlation between student confidence and mean facility, higher confidence for boys than for girls and lower confidence for students classified as socioeconomically disadvantaged, even after accounting for facility. We found that confidence was lower for older students, and this was particularly marked across the primary to secondary school transition. An important feature of the online platform used is that, when students answer a question incorrectly, they are presented with an analogous question about 3 weeks later. We exploited this feature to obtain the first evidence in an authentic school mathematics context for the "hypercorrection effect" (Butterfield & Metcalfe J EXP PSYCHOL 27:1491-1494, 2001), which is the observation that errors made with higher confidence are more likely to be corrected. These findings have implications for classroom practices that have the potential to support more effective and efficient learning of mathematics.