Predicting A-level grades accurately ‘near-impossible task’

For the working paper published today (Tuesday 11 August 2020), academics from UCL Centre for Education Policy & Equalising Opportunities (CEPEO) and Oxford Brookes Business School studied data from 238,898 pupils’ GCSE performance to see whether they could accurately predict their subsequent A-level results.

For the first time, they found that even by removing any opportunity for bias – and running additional checks on pupils’ gender, ethnicity and socioeconomic status – they could only predict one in four pupils’ best three A-levels correctly.

The researchers say the disruption COVID-19 has brought on formal examinations this year – with pupils instead assigned calculated grades by their teachers, which are then moderated by exam boards – highlights a wider problem with the general UK system.

The paper also shows that high-achieving comprehensive school pupils are more likely to be under-predicted compared to their grammar and private school counterparts.

Among high achievers, where under-prediction is most common, the team found 23% of comprehensive school pupils were underpredicted by two or more grades compared to just 11% of grammar and private school pupils.

One of the paper’s authors, and CEPEO Director, Professor Lindsey Macmillan (UCL Institute of Education), said: “This research raises the question of why we use predicted grades at such a crucial part of our education system.

“This isn’t teachers’ fault – it’s a near-impossible task. Most worryingly there are implications for equity, as pupils in comprehensives are harder to predict.

“Our work shows that these pupils have more noisy trajectories from GCSE to A-level. If you’re a straight-A student at a grammar or private school, you’re more likely to continue that to A-levels. But this research is telling us there’s a lot more movement around the grades between the two exam levels for comprehensive students.”

Teacher predictions of pupil grades are a fundamental feature of the English education system, forming the basis of students’ university applications and determining the wider life chances of pupils in post-secondary education.

Research by one of the authors, CEPEO Deputy Director Dr Gill Wyness (UCL Institute of Education), earlier this year showed that only 16% of university applicants were correctly predicted across their best three A-levels, when comparing teachers’ predictions to students’ actual grades. Of the rest, 75% were overpredicted and just 8% underpredicted.

The same study found that the grades of high-achieving students from low socio-economic backgrounds were more likely to be underpredicted.

For this working paper, the researchers used statistical and machine learning approaches to analyse detailed administrative data on prior achievement as well as demographic and school-level information.

They found their modelling only made modest improvements to the accuracy of teacher predictions, increasing the success rate from one in five to one in four pupils’ grades correctly predicted.

Predictions were improved by including data on ‘related’ GCSEs – those A-level subjects that have an equivalent GCSE – showing that exam subjects themselves need to be taken into account alongside student achievement and school type.

Maths was easier to predict among high achievers than other subjects such as history and chemistry, but for average and low achievers, the opposite was true.

English Literature was most accurately predicted across all achievement levels, while Law predictions were the least accurate.

For subjects without related GCSEs, the task was even more challenging, with lower prediction rates across the board.

The researchers say only moving towards a post-qualification applications and admissions (PQA) system would help remove potential inequalities.

Co-author Dr Wyness said: “We definitely don’t think teacher predictions should be replaced by computer predictions – this research serves to highlight the difficulty faced by teachers, and provides further evidence that the UK’s predicted grades system should be re-examined.”

The research was based on National Pupil Database records for a cohort of state and privately educated pupils who took their A-levels in 2008.

Links

• Working paper available at EconPapers
• Professor Lindsey Macmillan's academic profile
• Dr Gill Wyness' academic profile
• Dr Jake Anders' academic profile
• UCL Institute of Education
• UCL Centre for Education Policy & Equalising Opportunities (CEPEO)

Image

Photo by Ben Mullins on Unsplash

Media contact

Sophie Vinter

Tel: +44 (0)20 3108 7787
Email: s.vinter [at] ucl.ac.uk