The dangers of calculating a ‘Subject Progress 8’ score
November 1, 2016
By: Becky St.John, Principal Consultant
Here at SISRA, we are currently receiving a few enquiries about how to calculate a ‘Subject Progress 8′ score. I have some serious doubts about the validity of such a score, and if your school is using it, I would urge that it is treated with caution. We could calculate one, but does that mean that we should? It can be done for Maths, since it populates its own individual slots, but for the other baskets I’m not so sure.
For a start, let’s have a look at the ways that a subject-specific Attainment 8 estimate for the EBacc and Open baskets could be calculated:
- Overall A8 divided by 10 (2 x English, 2 x Maths, 3 x EBacc and 3 x Open)
- Element A8 divided by 3
- Element A8 divided by national average number of slots filled
And now let’s have a look at an example. If we calculated the 2015 EBacc subject estimate by dividing the EBacc estimate by 3, this means that students with a 4b would be expected to attain a high D. According to the national transition matrices in the RAISE Online library, in 2015 95% of Physics students nationally actually attained a D, while only 68% of Computer Science students did.
We have the same issue with the Open basket. For simplicity I have again looked at 4b students gaining a D. The variance between the attainment of students with the same starting point in different qualifications means that the expectation of the English Language or Art teacher that their 4b students will attain a D is relatively realistic, but to expect the same of the Business or ICT teacher is not in any way fair. Cutting the cloth differently, by calculating the estimate via either of the other options I mentioned above doesn’t help – the whole idea is inherently flawed.
This is compounded by the fact that we don’t actually know, on a national basis, how each qualification has contributed to the calculation of the overall or element ‘estimate’.
Imagine a student has a B in each of four EBacc subjects. According to the DfE the three of those subjects that contribute to the basket are selected by ‘result number’ which is allocated at the time. Basically it is arbitrary, so we can never know how any qualification contributes to the EBacc basket on a national level, again making it less than fair to apply equal expectations to the students and teachers studying each one. In my capacity as a Data Manager in school, I was advised by my contact in the DfE School Performance Data Unit that due to the way subjects are selected for each basket, is is not advisable to calculate subject specific A8 scores.
Lastly, note how volatile the Attainment 8 estimates are. They currently change from year to year, to such an extent that any projections for the following year are highly unreliable.
The DfE has issued no guidance on how to calculate a subject Progress 8 score, because it is basically incalculable. If you wish to try, please ask the DfE to specify how it should be done. If they can provide a fair and realistic way of calculating it, we’ll do it!