Schools that work for everyone: Measuring disadvantage

The UK government has recently been consulting on new proposals for ensuring good school places and opportunity for all young people in England, regardless of disadvantage (the consultation closed earlier this week). In my previous post I argued that any such policy should be based on a broad concept of disadvantage, taking account of economic, social and cultural capitals.

However, to achieve this in practice it is necessary to find suitable measures – a topic I will explore in this second post in the series.

Free School Meals

The Department for Education (DfE)’s main way of identifying disadvantaged children is those in receipt of free school meals (FSM) or who have been recipients at any given point in the previous six years (FSM6). (Although additional support is also available for other groups, such as children who have been in care).

At school-level, disadvantage is generally evaluated as the proportion of FSM6 children per school, year or class. The main advantage of this measure is that it is easy to compute and relies on a simple piece of information that is already available from schools and local authorities. However, as a measure of disadvantage it has three major shortcomings.

Firstly, it groups all children into just two broad categories (FSM or not), not acknowledging that educational chances vary incrementally across the socioeconomic spectrum. As a consequence, FSM often fails to identify children in  families marginally above the eligibility threshold and families that are just about managing but don’t qualify for benefits (so-called “JAMs”) as disadvantaged.

Secondly, it relies on income-level and being in receipt of benefits as proxies for poverty, with many earning more than £16,190 per year not being eligible. However, income level per se is not an adequate characterisation of economic capital, since factors such as housing or childcare expenses, number of children in the household or even regional variations have to be taken into account.

Thirdly, it disregards the other important forms of disadvantage discussed in my previous post, particularly children whose lack of educational capital is mainly due to a shortage of social or cultural capital.

The index of economic, social and cultural status (ESCS)

As part of the Programme for International Student Assessment (PISA), the Organisation for Economic Co-operation and Development (OECD) has developed an alternative measure of educational capital, the index of economic, social and cultural status (ESCS). Children scoring below the OECD average for ESCS are classified as being disadvantaged.

ESCS is calculated as a weighted average of three indices, generated from PISA surveys:

1. HISEI – the highest occupational status of a student’s parents.
2. ISCED (PARED) – the highest educational level of a student’s parents (converted into years of education).
3. HOMEPOS – an index of home possessions (measures the availability of 17 household items that act as measurements of family wealth and also the number of books at home).

ESCS addresses many weaknesses of FSM, and is acknowledged as a culturally sensitive measure of socioeconomic background that can be applied on a worldwide scale. It takes sociocultural factors into account when measuring disadvantage, and also does a better job of measuring levels of economic capital, since HOMEPOS is generally a more reliable indicator of how wealthy or deprived a family is than income level alone.

How do ESCS and FSM/FSM6 compare in practice?

Comparing ESCS and FSM directly is difficult. It’s like trying to link a person’s weight with their categorisation as overweight or not. You do not know someone’s weight simply by knowing whether or not they are overweight, and you’re not able to determine whether a person is overweight solely on the basis of their weight (you need to know their height as well).

Nevertheless, there is evidence that ESCS and FSM can both be used to predict educational achievement with some degree of success. Given its strengths, it should not come as a surprise that ESCS is the better predictor of the two. In fact, analysis by NFER statisticians suggests ESCS explains up to twice the variation in the results of all PISA subjects in 2009 and 2012 compared to FSM or FSM6.

This is the reason why new NFER research on the impact of disadvantage on maths performance, being published tomorrow, has used ESCS to paint a much more nuanced picture.

Towards a new measure

There has nevertheless been some criticism of the underlying methodology of ESCS, namely that the information provided by children on their household possessions and parents’ occupation may not be reliable. Another practical concern is that implementing ESCS beyond the scope of the relatively small PISA cohort would be technically challenging and costly, since it would imply conducting surveys and analysing data on a nationwide scale.

An ideal criterion for identifying disadvantage in a school context would therefore combine the best features of both FSM and ESCS: it would rely on data that is readily available from schools and local authorities while incorporating information on all three dimensions of educational capital. It would also provide a scale on which it was possible to evaluate the relative position of each child/school/country on the advantage/disadvantage spectrum.

One interesting approach proposed by NFER’s Dr Ben Styles is the creation of a set of three indices that combined data available from the National Pupil Database and Census Records to characterise the social and cultural backgrounds of each child’s area of residence in order to fine tune the FSM criterion. Dr Styles’ indices improved on the free school meals criteria by being significantly better predictors of outcomes such as student post-16 participation, retention, and attainment.

Educational inequalities hinder children’s chances to do well in school from very early on in their lives, even before they start school, and the gap between the more and less advantaged widens throughout the primary and secondary school years. Identifying inequality as early as possible allows for timely intervention, and so doing this effectively should be a priority.

OECD’s ESCS and Dr Ben Styles’ indices are alternatives to FSM that do a better job of identifying children that are disadvantaged in educational terms, or at least indicate possible ways of designing more accurate measurements and proxies. The feasibility of using these measures should therefore be investigated further.

In my third and final post of this series, I will examine how, equipped with better measures of disadvantage, we can use these to select and direct interventions most likely to make a difference to the children who most need it.