Spotlight on multi-academy trusts: Pupil outcomes

Just before Christmas, I was invited to present at a conference on establishing or joining multi-academy trusts (MATs). My role in the event was to share what the research tells us so far about MATs and I was given two questions to answer:

• What does the research say about the success of the MAT movement?
• Has enough research been conducted to present tangible arguments about the benefits of MATs?

These are known in research circles as leading questions! However, I accepted the challenge of answering them and thought it might be useful to share my thoughts a little more widely…

It will take more than one blog post to answer these questions though. Today, I will address what we know so far about outcomes for pupils in MATs. Next week, I’ll be looking at the question of whether enough research has been conducted and whether it has looked at the right things. Later in the week, I will be looking at the headline news from the DfE’s newest Statistical First Release (SFR) on MAT performance, which is being published on Thursday 25 Jan. In the final part of this MATs series, my colleague Jens Van den Brande will be delving further into the SFR data to see if that tells us more about these questions.

In September 2016, I presented to the Education Select Committee’s inquiry into multi-academy trusts. At the oral evidence session, I told the Committee: “the plans to expand MATs are racing ahead of the evidence and as researchers we are trying to keep up.” Sixteen months later, I don’t think a huge amount has changed. The number of academies and the number of MATs has gone up slightly, but the evidence is still lacking.

That is not due to a lack of desire for evidence. Leora Cruddas, the new CEO of Freedom and Autonomy for Schools – National Association (FASNA), recently laid down a challenge to the academic community to help find out more about the extent to which the most successful MATs are improving outcomes for children and closing gaps for the most disadvantaged.

The problem is good research takes time. This is particularly true for systems research. There is no point in looking at brand new systems because all you will capture is, at best, a baseline or, at worst, a snapshot of the preceding system. In an ideal world, a small part of the system would have been changed, we could have evaluated it, fed into an improvement loop that found the best ways to do things, and then spread what works. But policy making rarely works like this, and when the life chances of young people are at stake sometimes you can’t afford to wait that long to make changes.

So researchers do what they can. We are now getting to a point where enough schools have converted to academies, and have been in multi-academy trusts for a significant amount of time that we can tentatively start drawing research conclusions. So let’s have a look at the different data that is available.

Firstly, there’s Ofsted data. Luke Tryl recently highlighted that as with research, inspection legislation hasn’t kept pace with the creation and expansion of MATs. Ofsted therefore wants to change inspection legislation, so that it is allowed to inspect multi-academy trusts. We could look at Ofsted judgements at an individual academy level but broadly, this tells us more about the type of school that converted to become an academy rather than the current academy let alone the MAT (see for example DfE or NFER analysis).

Pupil performance data has the same caveats. Experimental statistics analysing MAT performance measures released by DfE in January 2017 found a varied picture. At Key Stage 2, more than half of the MATs had above average progress in writing and maths but on the measure of reading progress, over half of the MATs have scores that are below average. At Key Stage 4, two thirds of the MATs had Progress 8 scores that were below average (but the measure does not fully account for the historic performance of schools, including the poor prior performance of schools that became sponsored academies, which make more than three-quarters of the academies in these results).

If you compare MATs to Local Authorities you also see a very varied picture. At primary level, MATs are over-represented among both the best and worst performers, whilst at secondary level MATs make up a disproportionate number of the lowest-performing school groups. If you focus on disadvantaged pupils, you see very significant variation in outcomes for disadvantaged pupils, both between and within MATs.

Yet we need to know – with some urgency I would argue – how the most successful MATs are improving outcomes for children. Ambition School Leadership have started looking at this and published their initial findings in November 2017 but were unable to isolate a clear relationship between pupil progress and geography (isolation) or between performance and school type.

So, more research is needed! We also need clearer evidence on the best size of MATs, to know more about the best structure for MATs, and to know about the leadership, the organisation and the funding. We need to rise to Leora Cruddas’ challenge and, as she suggests, be much more intellectually curious about the world of multi-academy trusts.

Look out for part 2 in this MATs series, which looks at teacher career paths, school collaboration and financial efficiencies.