Revisiting free school meal eligibility as a proxy for pupil socio-economic deprivation
Abstract
Whether someone has ever had free school meal (FSM) eligibility over a six-year period is the measure of socio-economic disadvantage currently used in the English school system. It is used to monitor the socio-economic gap in achievement in the education system, to identify particular children at risk of low achievement and to direct funding to particular children and schools. In this paper we assess how well this measure predicts pupil attainment in secondary school in comparison to other measures of socio-economic background known to influence pupil attainment, such as parental education or income. We ask whether the FSM measure is an adequate proxy for a pupil's socio-economic disadvantage in an educational context. To do this we draw on the Longitudinal Study of Young People in England and matched administrative data. We find that the FSM eligibility measure correlates highly with other measures of socio-economic disadvantage, however it does not identify all children living in what would be deemed deprived households. We then compare the extent to which the FSM eligibility measure predicts educational achievement relative to other measures of deprivation and find that its predictive power is only marginally lower than many richer survey measures. This provides some reassurance on its use in policy.
Introduction
The gap in educational achievement between pupils from different socio-economic backgrounds remains an important unresolved problem in educational systems worldwide. Whilst the magnitude of socio-economic achievement gaps varies across countries and over time (Strand, 1999; Machin & Vignoles, 2004; Hertz et al., 2007; Ermisch et al., 2012; Jerrim, 2012; Jerrim & Micklewright, 2012), what is evident is that these gaps emerge in the earliest years of children's lives, persist into adolescence (Entwisle et al., 2007; Strand, 2008, 2014a,b) and impact well into later life (Reardon & Bischoff, 2011; Anders & Jerrim, 2012). In this paper we focus on England, where there are substantial differences in educational achievement between poor and rich pupils. Successive governments have emphasised the need to narrow this socio-economic gap in attainment to improve social mobility (Child Poverty Act, 2010; House of Commons Education Committee, 2014; Social Mobility and Child Poverty Commission, 2015). Numerous policies have been implemented to this end, with varying degrees of success. For example, schools receive additional funding for pupils from disadvantaged households, in an attempt to mitigate the impact of such circumstances on pupil achievement (Department for Education, 2013). All these efforts rely on having a good measure of the socio-economic disadvantage of a child.
There are a number of different ways in which children's family background may influence their academic outcomes, and as a consequence various indicators of pupils' socio-economic background have been found to be correlated with, and in some instance causally related to, differences in educational attainment. In particular, household income (e.g., Blanden & Gregg, 2004; Cooper & Stewart, 2013), parental education (e.g., Davis-Kean, 2005; Chowdry et al., 2012) and parental occupation (e.g., Letourneau et al., 2011) have all been found to impact on children's educational attainment.
However, in most education systems, teachers, school leaders and policy-makers do not have good information on all of these aspects of pupils' background and instead they rely on proxy indicators of the socio-economic circumstances of pupils. This is a crucial problem: without accurate information as to which children live in socio-economically deprived circumstances, the policy initiatives and programmes aimed at compensating for this and increasing pupil attainment may not target the right children. The reliable measurement of socio-economic deprivation is therefore essential, for three main reasons. First, to develop system-level policies to narrow the socio-economic achievement gap, there needs to be careful monitoring based on reliable indicators of both pupils' socio-economic circumstances and their achievement. Second, schools need to identify children particularly at risk of underachievement to provide them with support, and for this they also need a reliable indicator of socio-economic deprivation. Third, the funding system in England is compensatory, providing additional resources for pupils and schools in deprived circumstances. Again this requires a reliable measure of pupils’ socio-economic circumstances.
The aim of this paper is therefore to ascertain which measure of pupil socio-economic deprivation is most appropriate for use in policy- and decision-making in secondary schools in England. In particular, we will explore the robustness of the current measure, free school meal eligibility in the six years preceding a relevant educational stage (FSM ever-6), and compare this to a number of other direct measures and proxies for socio-economic deprivation. We will do so using rich survey data from the Longitudinal Study of Young People in England (LSYPE), the National Pupil Database and Census data for a cohort of children sitting secondary school examinations (GCSEs) in 2006.
When discussing FSM eligibility from this point onwards, we refer to the eligibility over a six-year period (i.e., over the six years prior to children in our sample sitting GCSE exams). For pupils to qualify for FSM, households must be claiming one of several state benefits, such as Unemployment Benefit or Income Support, and notify the school of this (Department for Education, 2014). More specifically, during the period when the children in our sample were completing secondary schooling, to qualify for FSM their families had to be claiming one of several benefits: (a) Income Support; (b) income-based Jobseekers' Allowance; (c) support under Part VI of the Immigration and Asylum Act 1999; (d) Child Tax Credit, as long as they were not entitled to Working Tax Credit and the household had an annual income under a particular threshold (which changed with inflation every year); or (e) the guarantee element of the State Pension Credit. In addition, since 2014, all infant pupils have been eligible for FSM, regardless of household circumstances.
It is important to note that these criteria do change over time: in particular, eligibility for Income Support has changed over the last decade or so. Further, there has been an updating of income thresholds, and the recent addition of Universal Credit to the list of allowable benefits. Hence the types of household measured by FSM eligibility also changes over time, and it is possible that the effectiveness of the measure in identifying children living in deprived circumstances will differ going forward as eligibility criteria change further. Previous research has also noted that FSM eligibility is dependent on the nature of the economic cycle, and therefore the proportion of pupils eligible for FSM will increase in a recession (Hobbs & Vignoles, 2009). Despite these fluctuations, however, for more than a decade, the FSM indicator has captured roughly 15–20% of the pupil population (as FSM eligible in the most recent year) and the Ever-6 version has recently become one of the main measures used to identify children eligible for the Pupil Premium (additional compensatory funding that schools receive to tackle the socio-economic achievement gap).
Despite being widely used, FSM eligibility has been criticised as an indicator for failing to adequately identify a range of socio-economically deprived groups, such as children in ‘working poor’ households (Hobbs & Vignoles, 2009) or families who choose not to claim the free meal for dietary, cultural or other reasons (Iniesta-Martinez & Evans, 2012; Lord et al., 2013). Specifically, Hobbs and Vignoles (2009) showed that FSM eligibility was not as effective at identifying pupils in low-income working households as it is those living in out-of-work households. Similarly, Kounali et al. (2008) showed that FSM eligibility did not accurately identify all children living in households where parents had low-status occupations, or were working part-time. Further, FSM eligibility is not a fixed characteristic of a given pupil (Department for Children, Schools and Families, 2009), as it is linked to parental employment, which in turn is linked to the economic cycle. Yet some characteristics of socio-economically disadvantaged families, such as low parental education levels, are persistent even if the parent moves into and out of (low-paid) employment. The cyclical nature of FSM eligibility and the need to identify families on the cusp of disadvantage are why pupils' FSM histories over six years are (currently) the preferred measure of pupil deprivation used by the UK Government. However, there is some evidence (Treadaway, 2014) that FSM eligibility at any point in pupils' educational trajectory is associated with an educational profile similar to that of pupils only captured by the six-year measure.
Research aims and questions
We contribute to this existing literature on the quality of the FSM indicator by (i) examining what it is that FSM measures in terms of its relationship with other (well-established) measures of socio-economic status that are important for educational attainment; (ii) comparing FSM against these measures to see how it ‘performs’ in terms of predicting educational attainment, with a view that if something performs ‘better’ then it might warrant consideration for use instead of FSM. The paper proceeds as follows: in the next section we discuss the use of different measures to identify children living in socio-economically deprived circumstances, we then go on to describe our data, analytical approach and present our results. We end with a discussion of the implications of our findings for policy.
Measures of socio-economic background and their relationship to attainment
The literature on educational inequalities and their sources differentiates between causal factors, which genuinely cause pupils' attainment to be lower or higher, and proxy factors, which are measures of socio-economic background correlated to the causal factors above. The latter may, however, also have some causal impact on achievement.
The causal factors consist mainly of household characteristics that affect pupils' chances of academic success. One key indicator on which there is evidence of a causal relationship with pupils' achievement is household earnings, or household income (see Blanden & Gregg, 2004; Plewis & Kallis, 2008; Walker & Zhu, 2011 for a discussion of the UK evidence). There is strong theoretical justification for why household income might causally influence pupils' achievement level. One line of argument is that households with more income are able to either directly purchase more educational inputs, or buy other assistance in the home that enables the parent to spend more time with and provide more help to their child. A recent review from Cooper and Stewart (2013) also concluded that income per se, instead of just the parental characteristics correlated with income such as education level, influences children's achievement and hence reductions in income, such as those caused by reducing benefits, will impact negatively on children's outcomes.
Another indicator with evidence of a causal impact is parental education level (e.g., Chevalier, 2004; Walker & Zhu, 2009 for the UK). The reason for this rests on the greater likelihood that parents with higher levels of education are more involved in their children's educational lives (Jeynes, 2007; Higgins & Katsipataki, 2015), are able to assist their children in their school work (Bower and Griffin, 2011) or may set higher educational aspirations and expectations (Social Mobility and Child Poverty Commission, 2015). All of the above may lead to better educational outcomes for pupils (Sirin, 2005; Hattie, 2009). Parental education may moderate the extent to which genetic factors manifest in pupils' learning (e.g., Friend et al., 2009 on the heritability of high reading ability in a twins-based study); this recognises the importance of heritability of traits from parents to children, but also their interaction with environmental factors (but see Polderman et al., 2015). Furthermore, there is also evidence that parents' occupation and education each have independent effects on pupils' outcomes (Bukodi & Goldthorpe, 2012).
Proxy measures, by contrast, are a pragmatic solution to the problem of measuring, at scale, some of these causal factors. Proxy measures include indicators which are more likely to be correlational rather than causal in nature. For example, various household characteristics have been shown to be associated with pupils' attainment. Single parenthood is one such indicator (Chevalier & Lanot, 2002); housing tenure is another (Boehm & Schlottmann, 1999; Jacob, 2003; though see contrasting evidence from Switzerland in Bourassa et al., 2016). While there may be a conceptual reason why single parenthood could be causally related to lower attainment for pupils (via the limitations imposed on time spent with children, for instance), the evidence also suggests that single parenthood is associated with household income, and will therefore represent a proxy for socio-economic background, rather than necessarily a causally linked factor.
Moving away from individual or household proxies for deprivation, one might also consider proxies that relate to the wider circumstances of pupils, capturing the characteristics of the neighbourhoods that pupils live in. For instance, the Index of Multiple Deprivation (IMD) and the Income Deprivation Affecting Children Index (IDACI) are two neighbourhood-level measures of deprivation that have previously been used to supplement FSM eligibility as a measure of socio-economic background (Chowdry et al., 2012) and are used as proxies for economic disadvantage by some local authorities in England.1 Other neighbourhood-related proxy measures refer to the educational and occupational profile of the communities in which pupils live: neighbourhood levels of participation in higher education and neighbourhood distribution of inhabitants in different occupational classes may also serve as proxies for individual-level socio-economic circumstances and drive social processes leading to other outcomes (e.g., Crane, 1991). These kinds of geographic data refer to areas rather than individuals, and there is evidence that they can misidentify pupils living in highly polarised regions, especially where small areas display large internal variations in the economic features of their inhabitants (e.g., Gorard, 2012). They may also fail to capture the differences between areas: neighbourhoods with similar characteristics in terms of housing tenure, for instance, may have very different conditions in terms of access to social services, schools and other educational opportunities. Nonetheless, the fact that neighbourhood measures have been (and are) used means that their comparison to individual-level proxy measures is relevant.
FSM eligibility is, however, the most widely used proxy in the UK context. Our purpose in this paper is to ascertain how FSM eligibility compares to other measures of socio-economic background, both causal factors and proxies, in terms of predicting pupils’ subsequent academic achievement. Even if FSM Ever-6 eligibility is a potentially flawed proxy measure of socio-economic background, as discussed above, it is important to determine empirically whether it is more or less flawed than alternative ways of identifying pupils from socio-economically disadvantaged backgrounds.
Prior attainment
We know that the best indicator of future academic achievement is prior attainment (e.g., Chowdry et al., 2012). Controlling for prior attainment (which is common practice in school effectiveness research, see Leckie, 2009; Baker et al., 2014; Strand, 2014a) shifts the analytical focus from explaining differences in absolute attainment to explaining the progress in achievement between the age at which prior attainment is measured and age 16 in our case (e.g., Strand, 2014b). Although we are interested in children's absolute level of achievement, given the above research, which suggests it is strongly correlated with their socio-economic circumstances, we also need to look at the progress children make within their school. Indeed, from a policy perspective, the primary use of the FSM eligibility measure is arguably to determine the progress made by socio-economically disadvantaged pupils relative to others. Therefore, although we initially present results from analysis that does not include a measure of prior attainment, we then include prior attainment in our analytical models to explore the performance of our measures of deprivation in relation to predicting pupils’ progress.
We also recognise that prior attainment could potentially be used as an alternative indicator for pupils at risk of low achievement irrespective of social background, as distinct from identifying pupils from lower socio-economic backgrounds. This approach is not without its problems, and we return to it in our final discussion section, where we consider the policy implications and difficulties associated with targeting resources on the basis of prior attainment as distinct from targeting on the basis of socio-economic deprivation.
Data
Sample
We use the first LSYPE (LSYPE1) for the analysis. To maximise the information available, we primarily rely on the first wave of LSYPE1. The initial sample consisted of 15,770 young people in England who consented to the matching of their original survey responses to administrative data records from the National Pupil Database (NPD). LSYPE1 survey data were therefore linked to individual-level administrative data drawn from the NPD. Additionally we also linked the LSYPE1 data to neighbourhood-level data from the 2001 Census, using the neighbourhoods where LSYPE1 participants lived at the time of the first survey wave.
Owing to survey attrition, the original sample only contained NPD attainment records for the pupils still in the survey at the time of their Key Stage 4 examinations. However, we were able to merge in an additional Key Stage 4 data extract, prepared specifically for this analysis by the Department for Education, which added 2934 Key Stage 4 achievement records for the pupils who had consented to NPD matching before attriting from the LSYPE1 study.
To allow comparability between proxy measures, our analytical approach was based on estimating a set of models on one complete-case sample, meaning that only those cases with no missing data on any variables in our models were retained. To mitigate the risk that such a strategy would yield a biased sample, we used Department for Education-provided imputed variables (Piesse & Kalton, 2009) where the proportion of missing data would have reduced the size of our estimation sample. We imposed an additional restriction on our estimation sample and removed all young people in special schools or pupil referral units at Key Stage 4. We took this approach as these institutions are atypical both in terms of the intake of pupils and in the likely relationship between achievement and family background measures. The final analysis sample consisted of 12,678 individuals in 358 schools, who had complete (or imputed) data on all of their survey measures, as well as full information from the administrative data sources.2
Outcome measure
The outcome measure was the total capped GCSE point score, a measure known as ‘Best 8’ (now superseded by ‘Progress 8’; Department for Education, 2015), as it stood in 2006. This was a total score formed of the best eight GCSE grades a pupil achieves, including equivalent qualifications (i.e., qualifications deemed equivalent to a GCSE in 2006). We opted to use the raw measure as this allowed for the straightforward interpretation of coefficients into GCSE grades. The GCSE scoring system in place at this time was 16 points for a grade G and an additional 6 points for each higher grade, up to 52 points for an A and 58 points for an A*. Therefore, we interpret a difference of 6 points as a one-letter-grade difference on a single GCSE. A difference of 12 points, for example, would translate into either a two-letter-grade difference on one GCSE or a one-letter-grade difference on two separate exams.
Control measures
To isolate the relationship between socio-economic background and pupil achievement, we needed to account for confounding factors that influence pupil achievement and that may be correlated with socio-economic background, irrespective of the measure used. To do this and to allow comparisons between models, we used a consistent set of control variables for all models, drawn from the existing literature on the determinants of pupil achievement. These controls recognise, for instance, the variation in pupil outcomes by ethnicity (Strand, 2008, 2010, 2011, 2012), geography (Hamnett & Butler, 2011) and school type (e.g., Hattie, 2009; Gibbons & Telhaj, 2011). The controls therefore relate to individual pupils, the schools they attended, as well as the region in which they lived when first interviewed for LSYPE1.3
Measures of socio-economic background
We relied on a review of the literature (Sutherland et al., 2015a) to select measures of socio-economic disadvantage with substantial evidence about their potential to both capture socio-economic background and predict educational attainment. The choice of measures consisted of both household characteristics and neighbourhood features, as described in what follows.
FSM eligibility was our reference indicator, measured as closely as possible to the Ever-6 version used by the government: given that LSYPE1 participants sat GCSE exams in 2006, the NPD extract contained FSM eligibility records going back to only 2001, and therefore we used eligibility for FSM at any point in the five years prior to GCSE exams.
For the household qualifications measure we used an LSYPE1-imputed variable to derive a series of dummy variables, each identifying the highest qualification in each pupil's household, held by mother, father or carer. We applied a similar procedure to construct a set of household occupation indicators, also derived from LSYPE1 and drawing on the NS-SEC classification (Office of National Statistics, 2005a,b).
Two additional variables were created for a measure of household work-related characteristics: employment status (dummy variable identifying households where at least one of the parents was in full-time employment) and lone-parent family status (to account for the fact that single-parent households cannot have more than one person in employment).
To create the household income indicator we used the imputed LSYPE1 self-report measure.4 We acknowledge that self-reported income data may be of poor quality, as it can suffer from non-response bias and mis-estimation by respondents, which may adversely affect the predictive power of the income proxy variable. We return to this issue in our discussion of the results.
The household characteristics set was composed of mother's age (included as a proxy for non-traditional family arrangements, for instance old or young step-families), dummy variables for housing tenure (with the reference category being home ownership) and the number of people in the household during wave 1 of LSYPE1.
We carried out a set of preliminary analyses to select from the numerous neighbourhood-level proxy measures that our literature review initially identified, notwithstanding the problematic nature of their use, as noted earlier. Where two measures were very highly correlated we removed one, prioritising the measure that had better theoretical justification for inclusion. We considered IDACI, IMD, neighbourhood proportion in higher education, proportion with degrees and proportion in higher occupations measures. We retained variables identifying the proportion of people in the neighbourhood who were in the top two occupational (NS-SEC) classes (hereafter, top-level occupations) and the IDACI (rescaled to range between 0 and 100).
Analytical approach
Our analytical approach consisted of estimating a set of multi-level linear regression models using the estimation sample5 described above. Using multi-level models accounts for pupils’ clustering in schools, while also allowing pupil- and school-level characteristics to be included together (Clarke et al., 2015). Multi-level models also allow for between- and within-school variance in pupil outcomes to be assessed. Our focus here is on the within-school variance and its association with measures of socio-economic status, not on the relative effectiveness of schools to tackle socio-economic attainment gaps as measured by variation across schools.
To address our empirical question as to which measure of socio-economic status would best capture differences in Key Stage 4 outcomes, we estimated separate multi-level linear regression models, substituting each of the measures whilst including all the other control variables. This allowed us to ascertain which model, and by virtue of the common estimation sample and common controls which measure of socio-economic background, explained the highest proportion of variance in individual pupils’ attainment at Key Stage 4.
where Yij is a continuous outcome measure for individual i in school j. α0 is the overall intercept (average), x1ij is an individual-level measure for person i in school j and z2j is a school-level variable. εij and μ0j are, respectively, the pupil- and school-level error terms (residuals).
To compare the predictive power of each variable we measured the proportion of the variation between pupils that is explained by the particular measure, using the Snijders–Bosker indicator (Snijders & Bosker, 1994, p. 342). A higher proportion of variance explained suggests that the variable is more predictive of pupil attainment and hence a ‘better’ indicator. In what follows we report the Snijders–Bosker R2 for within-school variance for each of the models we estimate, and refer to it as the ‘proportion of explained variance’ in the individual-level outcome measure. This also reflects the overall aim to establish the best measure for deprivation in explaining individual-level, rather than school-level, variation in attainment.6
Limitations of our approach
There are of course some limitations to our analysis. As with any survey data set, LSYPE1 suffers from attrition and to the extent that the data is not fully representative of the English population of secondary-school pupils, our results may be biased. However, prior evidence on the use of the LSYPE1 data for educational research (Anders, 2012), and the additional Key Stage 4 data from the NPD which we merged in to limit the number of survey participants missing achievement data, would suggest that our sample is sufficiently robust to carry out these analyses.
Additionally, we focus on pupil attainment in secondary school for one cohort and clearly one question is whether FSM eligibility is also a good proxy for socio-economic background in primary school. There is substantial evidence (e.g., Leckie, 2009; Connelly et al., 2014) pointing towards the influence of early years and primary schooling on later educational achievement and the large effect socio-economic deprivation can have at those earlier stages too. It is therefore worth noting that a replication of this analysis carried out using the Millennium Cohort Study and focusing on Key Stage 2 attainment as the outcome measure (Sutherland et al., 2015b) suggested that the results described above generally hold for primary education.
There is also a literature that has debated the value of the use of standard errors in analyses of this type (Gorard, 2005). Though we adopt the widely used multi-level model approach which adjusts standard errors to allow for clustering of pupils within schools, we have validated our results using other approaches, including ordinary least squares (OLS) regression.7
Results
Do the potential proxy variables identify the same groups of pupils?
We first discuss the extent to which the different candidate measures identify different groups of pupils, and the overlap between the FSM ever-5 measure and the other indicators. For example, we can determine the proportion of pupils living in households with long-term unemployment that have ever been eligible for FSM; or conversely, the proportion of children eligible for FSM (in the five years preceding their GCSE exams) who live in households with long-term unemployment.
Our basic descriptive results (Table 1) indicate that 90% of pupils living in a household with parents who are long-term unemployed are eligible for FSM, and more than half of those with parents working in routine occupations are FSM eligible. Hence, there is a great deal of overlap between FSM eligibility and low parental occupation status. However, the second column of Table 1 also suggests that FSM-eligible pupils are not all found in households with the same type of occupation. In fact they are relatively evenly spread across semi-routine, routine and unemployed households (that said, nearly three-quarters of FSM-eligible pupils come from lower-supervisory to long-term-unemployed households). But what this does tell us is that the FSM measure captures pupils whose parents work in a range of occupations. This reflects, at least in part, the nature of the FSM eligibility criteria, whereby current parental unemployment is likely to result in FSM eligibility regardless of the type of parental employment.
| Proportion of pupils in each category who have been eligible for FSM in the five years preceding GCSEs | Proportion of FSM-eligible pupils (ever in the five years preceding GCSEs) found in each category | |
|---|---|---|
| Household occupation | ||
| Higher managerial occupations | 3.7% | 1.9% |
| Lower managerial occupations | 9.4% | 10.7% |
| Intermediate occupations | 16.1% | 6.9% |
| Small employer occupations | 19.1% | 8.2% |
| Lower supervisory occupations | 25.7% | 10.4% |
| Semi-routine occupations | 41.8% | 20.9% |
| Routine occupations | 62.9% | 18.7% |
| Never worked/long-term unemployment | 90.6% | 22.3% |
| Total 100% | ||
| Household qualification | ||
| Degree | 7.4% | 4.7% |
| HE, below degree | 10.2% | 6.1% |
| A-level or equivalent | 13.3% | 9.1% |
| GCSE A–C | 22.9% | 23.2% |
| Level 1 and below | 37.7% | 9.5% |
| Other | 46.2% | 2.7% |
| None | 62.4% | 44.7% |
| Total 100% | ||
| Household characteristics | ||
| Own house | 10.5% | 28.9% |
| Renting privately | 48.1% | 10.5% |
| Social housing (council or housing association) | 62.9% | 59.2% |
| Any other housing arrangement | 29.4% | 1.4% |
| Total 100% | ||
| Single parenthood | 53.2% | 49.3% |
| Both parents unemployed | 87.7% | 62.4% |
| Income (below 2006 FSM eligibility threshold) | 48.1% | 69.4% |
| Neighbourhood occupations | ||
| Proportion in top-level occupations < 25%a | 32.3% | 75.3% |
| Neighbourhood child deprivation | ||
| IDACI (bottom 25%)a | 55.1% | 52.0% |
Note
- a These proxies are entered as continuous variables in the models; the cut-off points are for presentational purposes only.
Further, nearly two-thirds of pupils in households where parents have no qualifications are eligible for FSM. Approximately half of all FSM pupils fall into this category of parental qualifications. Irrespective of other household characteristics, the FSM measure will therefore identify pupils whose parents have lower-level or no qualifications, with roughly 57% of all FSM-eligible pupils living in these types of household.
In terms of housing tenure, 63% of pupils in social housing (local-authority-owned housing, or renting from a housing association) are FSM eligible; conversely, nearly 60% of FSM-eligible pupils are found in council housing. The overlap between these two characteristics is therefore moderately large, but the two will not identify the exact same group of pupils and households.
Pupils’ FSM ever-5 eligibility is also strongly correlated with single parenthood, and with unemployment: 88% of pupils in a household where both parents were unemployed were eligible for FSM in the previous five years. The relationship between FSM eligibility and household income is less straightforward, despite FSM eligibility criteria including a low-income threshold. Approximately 69% of FSM-eligible pupils are classified as living in households with low income. This may of course be partially attributable to the difficulties in self-report measures of income but equally may reflect the fact that many households that do not experience very low income are nonetheless FSM eligible. Perhaps more problematically, only 48% of those in low-income households are eligible for FSM, and therefore more than half of those children who live in households with very low income and who are presumably therefore at risk of low achievement are still not eligible for FSM and therefore not identified as being in need of additional support. This reflects an issue with the FSM eligibility indicator previously raised in Hobbs and Vignoles (2009).
Lastly, the neighbourhood-based measures are not as strongly related to FSM ever-5 eligibility as other measures. Although a large proportion of individuals living in the bottom quartile of IDACI neighbourhoods (i.e., neighbourhoods with the highest rates of deprivation affecting children) are indeed FSM eligible, many FSM-eligible pupils do not live in this group of most deprived neighbourhoods. This preliminary evidence would suggest that were one to switch to using neighbourhood-based measures as proxies of individual socio-economic background, there would be a clear risk that these proxies would capture very different populations of children compared to the current FSM eligibility measure, potentially missing out on high numbers of pupils who live in disadvantaged households but in richer neighbourhoods.
To foreshadow our more detailed empirical assessment, it appears from this descriptive exercise that FSM is a single measure that captures a multidimensional group of pupils from a range of backgrounds.
What is the relative predictive power of each measure?
We first used the FSM ever-5 indicator as the proxy measure of socio-economic deprivation. This explained 23.3% of the within-school variance in Key Stage 4 scores. Being ever-eligible for FSM versus not, net of all other controls, was associated with a 56-point difference in the total capped GCSE score. This is a considerable difference, equivalent to non-FSM pupils gaining one grade better across seven GCSEs (e.g., moving from a C to a B) and two grades better on an eighth GCSE (nine letter grades in total).8 See Table 2.
| Model | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |
|---|---|---|---|---|---|---|---|---|---|
| Individual-level explained variance | 23.3% | 22.5% | 25.8% | 25.6% | 20.6% | 24.4% | 20.8% | 21.1% | 31.3% |
| Between-school variance | 1.3% | 1.5% | 0.9% | 1.0% | 1.1% | 1.4% | 1.5% | 1.1% | 0.9% |
| FSM eligibility, ever | |||||||||
| Ever-5 FSM eligible | −56.5a | −18.1a | |||||||
| Employment/single parent | |||||||||
| Min. one parent full-time | 41.4a | 5.5 | |||||||
| Single-parent household | −22.7a | −11.6a | |||||||
| Household qualifications | |||||||||
| HE, below degree level | −29.1a | −19.4a | |||||||
| A-level or equivalent | −38.2a | −21.9a | |||||||
| GCSE-level or equivalent | −55.3a | −29.5a | |||||||
| Other qualification | −67.9a | −33.8a | |||||||
| Level 1 qualification | −80.7a | −42.4a | |||||||
| No qualification | −91.7a | −49.5a | |||||||
| Imputation flag | −14.1a | −14.9a | |||||||
| Household occupations | |||||||||
| Lower managerial | −23.5a | −10.9 | |||||||
| Intermediate occupation | −34.3a | −8.3a | |||||||
| Small employers | −54.5a | −25.0a | |||||||
| Lower supervisory | −66.1a | −31.5a | |||||||
| Semi-routine | −70.5a | −24.3a | |||||||
| Routine | −91.4a | −34.4a | |||||||
| Long-term unemployed | −95.8a | −30.6a | |||||||
| Imputation flag | −3.8a | 0.159 | |||||||
| Household income | |||||||||
| Income (£1000s) | 0.5a | 0.07a | |||||||
| Imputation flag | −0.4a | −2.1 | |||||||
| Household characteristics | |||||||||
| Age of mother | 1.6a | 1.1a | |||||||
| Mother of working age | 58.0a | 29.6a | |||||||
| Housing tenure: private rent | −34.7a | −11.7a | |||||||
| Housing tenure: LA rent | −53.2a | −20.3a | |||||||
| Housing tenure: other | −32.4a | −21.8a | |||||||
| Household size (persons) | −2.2a | −3.6a | |||||||
| IDACI | |||||||||
| IDACI score | −1.1a | −0.08a | |||||||
| Neighbourhood occupations | |||||||||
| Prop. top-level occupation | 1.7a | 0.6a | |||||||
Note
- a p ≤ 0.05. Results for basic controls omitted from this table. Reference categories for proxy variables: Model 1, never eligible for FSM; Model 2, no parent employed full-time, household with both parents; Model 3, degree-level qualification; Model 4, higher-managerial occupation; Model 5, none (income—continuous variable); Model 6, house tenure owner-occupier, mother not of working age, all other variables continuous; Model 7, none (IDACI = continuous variable); Model 8, none (proportion higher occupations = continuous variable); Model 9, all of the above.
Next, we looked at household characteristics in relation to employment and single parenthood (22.5% individual-level variance explained). We found that compared to a two-parent household where none of the parents are in full-time employment, pupils in a household where at least one parent was working performed an average of 41 GCSE points better.
We then used the highest educational level of either parent (or carer) in the household as the measure of socio-economic status, and found a strong linear relationship between the different qualification categories and pupil attainment, with 25.8% of variance explained. Children in households with no qualifications, or only qualifications at level 1, were predicted to achieve 81–92 GCSE points fewer than pupils in households with degree-level qualifications. This was equivalent to almost a two-letter-grade difference across all eight GCSEs.
We observed a very similar pattern for household occupational class, with 25.6% of variance between pupils explained. In particular, we noted that pupils with parents in semi-routine, routine and unemployed households attained an average of between 70 and 96 points fewer than those in the highest occupation class (higher managerial). In terms of GSCE grades, this equates to a maximum of a two-letter-grade difference on every one of the eight GCSEs counted in the capped score.
We then substituted household income as our measure of socio-economic deprivation. The results suggest that an increase of £10,000 in the annual household income was associated with five additional GCSE points, less than one letter grade on a single paper. This weaker relationship, when compared to the previous measures of deprivation, was also reflected in the proportion of individual-level explained variance, at 20.6%. As noted earlier, there are issues with data quality and therefore the results of this model should be interpreted with those in mind.
Lastly, on individual-level measures of deprivation, we used a set of household characteristics, including housing tenure (explaining 24.4% of the within-school variance). Pupils in households that owned their dwelling were likely to attain 53 points more than pupils in households in social housing, a difference of more than one letter grade on all eight GCSEs in the capped score measure; for children in families that privately rented accommodation, the mean difference from children in social housing was 19 points, or roughly three letter grades on one single GCSE paper.
Neighbourhood-based proxy measures in general explained less variation across pupils than the FSM ever-5 indicator. Higher scores on the IDACI neighbourhood measure indicate worse levels of deprivation, and each additional point on the IDACI scale was associated with approximately one less GCSE point on average. This suggests a wide discrepancy between those at the extremes of the IDACI scale, equivalent to approximately two full GCSEs less for pupils in the most deprived neighbourhood compared to those in the least deprived one.
The proportion of working-age adults in the neighbourhood in managerial and professional occupational class categories was explored next. Every percentage point increase in this measure was associated with an increase of 1.7 GCSE points. This would mean that, for instance, pupils in a neighbourhood where 30% of adults were in these occupational classes performed 17 GCSE points better (i.e., almost three letter grades better on a single paper) than pupils in a neighbourhood with only 20% of adults in the same situation.
In addition to these single-measure models, we also estimated a further model, which included all the above candidate measures as well as the FSM ever-5 eligibility indicator. We did this for two reasons: first, to explore the combined explanatory power of all proxy measures, while recognising the fact that they are likely to often capture the same characteristics of pupils; and second, to determine whether FSM eligibility would remain a predictor of attainment even when included alongside all the other measures. As expected, the explanatory power of this all-measures model was higher than that of any previous model, at 31.3% of individual-level variance explained. Perhaps surprisingly, whilst the effect size was three times smaller (shrinking to an 18-point difference between FSM ever-5 eligible vs. not), FSM eligibility (ever-5) remained negatively correlated with Key Stage 4 outcomes. This suggests that FSM eligibility may capture something unique about the lived experience of deprivation. We should not forget that this does not represent a causal estimate—and it may be, for example, that the parents of children who declare to schools that their child is FSM eligible are motivated in part to do so because their children are already struggling at school.
Lastly, we ran a supplementary model that focused on progress, including a measure of prior attainment (not tabled). The proportion of explained variance was substantially higher for this model than for all others estimated in this analysis, at 44.3%.9 The FSM ever-5 measure retained its negative association with GCSE scores, with FSM ever-5 eligibility being associated with a 10 GCSE point difference compared to non-FSM-eligible pupils, net of all other socio-economic deprivation measures and of prior attainment. As expected, prior attainment is the strongest predictor of future attainment but as a measure of socio-economic disadvantage, ever-5 FSM continued to have an independent effect on attainment. The results concerning all the other measures of socio-economic deprivation were very similar in their association with both absolute attainment and progress in attainment, suggesting that our conclusions about them stand in both situations.
Conclusions and implications for policy
The purpose of this work was to establish whether pupils’ FSM eligibility represents a high-quality proxy indicator for pupils who are experiencing socio-economic deprivation. We explored whether a range of other alternative indicators, obtained from survey and administrative data, might be better at predicting pupil attainment at Key Stage 4, and hence more useful than FSM eligibility to identify pupils at risk of low achievement. These alternative indicators measured individual and household characteristics, as well as neighbourhood features.
Our results indicate that, from the measures tested, parental occupation levels and parental education are the best predictors of pupils’ attainment. These measures are more predictive than FSM eligibility, though the improvement is only marginal (in the region of two to three percentage points of individual-level variance). Household income is surprisingly weakly associated with pupil attainment, and this may well be because self-report measures of income are often prone to error. This implies that if we want to measure the socio-economic gap in achievement in the education system, or if we want to identify individual children at risk of low achievement, using parental education or occupation is likely to be better than FSM eligibility, though only marginally so.
Before we consider replacing FSM eligibility with measures of parental education and occupation as indicators of pupils’ socio-economic disadvantage, however, we need to consider the availability of such data at scale and the practicality of data collection. Parental qualification level and occupation would be difficult to collect across the entire education system, and any attempt to do so would place an burden on schools. Given this problem, one might be cautious about recommending replacing FSM eligibility, particularly given that the gain in predictive power is modest. However, it is important to note that a decision to continue to rely on the FSM indicator comes at some cost. As discussed in previous work published in this journal (Hobbs & Vignoles, 2009), there are many disadvantaged children who are not eligible for FSM, including the so-called working poor. Some alternative measures, such as parental education, better identify such children. The policy implications of this are, for example, that these children with low-educated parents and at risk of low achievement miss out on additional support or funding if it is targeted purely on the basis of FSM eligibility.
The other set of candidate indicators tested, namely neighbourhood-based measures, are not as good at predicting GCSE attainment as the FSM eligibility benchmark. These data have the advantage that they would be more readily available, as they form part of the Census. However, these variables are not as predictive of individual pupil attainment and would therefore not be good substitutes. So, although the characteristics of an area may impact directly on a child's attainment and hence these variables predict attainment, they do not identify individual children who experience socio-economic disadvantage. Table 1 illustrates that a fair proportion of the pupils currently identified as at a disadvantage (by the FSM eligibility measure) do not reside in neighbourhoods where occupational status is low and deprivation is high, a further argument against neighbourhood-based measures of socio-economic background. Targeting additional support for children on the basis of their neighbourhood would mean some deprived children living in richer neighbourhoods miss out, and some children who are not deprived but live in a deprived area would benefit.
There are of course other strategies that might be pursued in the search for the ‘best’ measure to predict pupil attainment. We have shown that the predictive power of the household-income measure is low, and this is most likely because of problems inherent in collecting self-reported earnings and income information. An alternative approach would be to use administrative data on parental income. A measure drawn from HMRC tax records, for example, would be accurate and therefore likely to be a better proxy indicator than FSM eligibility—in the absence of such data, however, measures of income that rely on self-reports are not likely to represent a better alternative to FSM eligibility.
Another alternative strategy is for schools and the Department for Education to use pupils’ prior attainment levels as a predictor to identify pupils at risk of low attainment in the future. Our analysis suggests that this would certainly be effective at predicting future attainment. However, using prior attainment as a proxy for socio-economic background is problematic for a number of reasons. First, on a conceptual level, low attainment is not synonymous with socio-economic deprivation. If the aim is to identify pupils who experience educational disadvantage because of their circumstances, we need to identify pupils who are socio-economically deprived not just those not doing well at school. Not all low-attaining pupils will be deprived and hence we may only want to direct additional resources (including the Pupil Premium) to pupils who are socio-economically disadvantaged. This raises some fundamental conceptual and indeed ethical issues about policy aims in this sphere. Second, at a practical level, when a child enters the school system their prior attainment is not known and testing very young children is fraught with difficulties. Therefore, in early primary school it would not be easy to identify pupils at risk of low attainment since there would not be good prior attainment measures.
Yet another potential solution is to combine the FSM measure with the neighbourhood proxies. As we have shown above, this would increase the predictive power of the models over and above any individual measure, and would enable us to identify pupils who are not FSM eligible but yet are living in deprived households and neighbourhoods. The limitations discussed above, pertaining to the neighbourhood measures, would however remain relevant.
Going forward, policy-makers need a measure that is consistent over time, able to identify children living in socio-economically deprived circumstances and that predicts low achievement. FSM eligibility does that currently but we are mindful that the FSM eligibility measure itself will change as the distinctions between those receiving Working Tax Credit and Child Tax Credit become blurred, and if these benefits are merged with the implementation of the Universal Credit system. Going forward, therefore, the measure may identify a different group of pupils, an issue which policy-makers and schools need to be very aware of.
The Education Select Committee (2014) also raised the issue that the FSM eligibility indicator divides pupils into two groups only and does not have the potential to differentiate between more or less wealthy families within each of those two groups. Understanding that there is variation within the FSM and non-FSM groups is crucial when reporting any differences in the attainment of these groups.
Lastly, we would like to note that it is very likely that the underlying relationship between pupils’ socio-economic background and their attainment is stronger than our results above would suggest. The relationship is mitigated somewhat by the compensatory and redistributive nature of the educational system already in place, which has seen schools with high proportions of FSM-eligible pupils receive additional funding in an attempt to provide these schools with the resources needed to compensate for the detrimental effects of deprivation. Were this not to be the case, we would have expected eligibility for FSM to be even more strongly, and negatively, associated with pupil attainment.
