The A Level Results Scandal – how not to use algorithms

A Level results are dropping into young people’s in boxes this morning and from what we have heard on the news and seen with the Scottish results there is going to be a lot of disappointment this morning.

Earlier this year, when it became clear that the normal summer exams would not be possible, teachers set to work providing recommended results. They based those recommendations on how students had done so far in terms of the work they submitted, predicted grades and no doubt some awareness of personal circumstances, temperament etc. Now, that might not be ideal, but it is better than anything and I’m sure that future employers will recognise that this was an unusual year.

However, the exam boards have applied an algorithm across schools in order to get what they believe will be truer, more consistent grades. No doubt, something like that will have been needed. Each year, adjustments are made, mainly to the grade levels to allow for quirks in the process and year on year differences. We cannot expect individual teachers to know the wider picture and of course each school will have been putting its best spin on things.

The problem is, that in a world where we “follow the science” computer algorithms have taken on a God like life of their own.

We should have already learnt from our experience of some of the models produced for coronavirus that these things are not infallible. They have their place, they help us to predict, to look forward and prepare for different scenarios, they can inform our decision making but they should not be used to make the final decision.  All that an algorithm can do is use existing data from past events to provide an idea of what future events might be like.

What should have happened, is that the algorithm should have been used much earlier in the process.  This would have enabled each school to know the range of grades that seemed likely based on past performance.  By the way, at this point we are recognising something that schools have been saying for ages, that the performance of a school is not just about how bright the students are and how good the teachers but on other factors such as social and economic conditions.  In any case, the schools would have then been able to compare the grading with the algorithm.  At that point they would have had a distribution curve of grades and been able to identify whether or not their grades were within tolerance.

It would have then been possible for the schools to identify those grades that looked to be outside of the expected norm and either revise them or provide further justification for the decisions made.  You see algorithms and computer models are meant to be used alongside other information to help humans with on the ground knowledge make informed decisions. They are not meant to overrule the human decision. 

Algorithms were made for man, not man for algorithms.

From an outsider’s view looking in (an outsider with experience of working with computer modelling and some legal training), it seems to me that there is a strong case to argue that students have been treated unfairly and that the examiners have acted outside of their authority in how they have gone about adjusting grades this year. I believe that there is a strong case for a Judicial Review of the process and that individual students would have the right to take their own cases through the courts to get their results returned to what they were assessed at.

One thought on “The A Level Results Scandal – how not to use algorithms

  1. I think that using this computer modelling was far from ideal, and it is never fair to judge an individual student based off the performance of their school. Plus this is rather obviously going to hit those on the lowest incomes the most, meaning they are less able to go to the uni they wish to, or will have to put in effort to appeal and possibly do the exam.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: