There’s been much social media discussion about this article from Fraser Nelson in the Spectator reporting a conversation he had with Prof Graham Medley the chair of SAGE. In the conversation, Medley indicates that SAGE tends to model the worst case scnarios because those are the ones that if they turn out to be true will need action.
This has been read by a lot of people as suggesting evidence that SAGE are biased and are deliberately skewing the data towards the worst outcomes in order to push the Government towards tougher restrictions.
I think that’s a misunderstanding of the situation and the crucial thing here is the distinction Medley makes between scenarios and predictions. We could also add a third category in “pojections.” If you want to know my view on how many people will be in hospital as a result of COVID, then I can give you a projection. I know that about 90,000 people have tested positive for COVID over the last few days. I also know that about hospitalisations equate to about 1.8% of cases from 10 days ago and that deaths account for 0.25% of cases from a month ago. So I can project forward that by Christmas we’ll be seeing 1600 admissions each day and that by early January we are looking at 183 deaths per day. Those are not hard prophecies because the % of admissions and deaths may change due to things like Vaccine Efficacy. So, people offering projections will still provide a range.
You can also ask me “how many cases, admissions and deaths” would we see in a certain scenario. For example, you can set a scenario where Omicron turns out to be extremely virulent and where vaccine efficacy against serious disease drops to 80% and against transmission to 25%. I can then tell you how many cases, admissions and deaths we’ll see. Well I could if I had a fancy modelling tool. I don’t but we can all work out that it would be pretty bad. That’s important because if things are going to be pretty bad, then we need to take action. The Government has a choice. Does it take action to try and prevent the pretty bad things happening (like through lockdowns, NPIS etc) or does it take action by preparing for the affects of the scenario. That might mean opening up nightingale hospitals, drafting in the military to keep supply chains open or asking doctors to make some very difficult treatment decisions.
Predictions are different. Really at that stage you are asking me how likely a particular scenario is to play out. These scenarios are too far into the future to simply rely on projections. That’s where we need to talk in terms of probability.
This last point is crucial because I think Medley is right to say that we run scenarios for the worst case scenarios. I mean, if your company said they were going to get everyone in to practice emergency planning, you wouldn’t be impressed if you gave up your Saturday to sit in the office where the only real drama was someone bringing in a birthday cake (though getting some cake might be a small mitigation). You expect all the worst case possibilities to be thrown at you, fire, flood, alien invasion, that kind of thing. And that’s what computer models are meant to do. They run through the war games virtually so we don’t have to recreate them in real life. I don’t need to prepare for a good day when the only drama is birthday cake. I’m already well equipped for that.
However, when it comes to making those decisions about which plans to deploy, the Government does need more than scenarios. It needs predictions. Now, it might be that SAGE are reluctant to tell us the probability of different scenarios and there’s good reason for that. You see, by sharing that information, they can affect human behaviour which actually changes the scenarios. If for example we were told last week that the actual probability of there being 5000 admissions a day and half a million cases was extremely low then we may have become complacent and not bothered to be careful about masking, socialising and getting our boosters. The result of that is that the probability of those scenarios could well have increased dramatically. So, these things are not easy when dealing with human behaviour.
It’s also worth adding that whilst SAGE should be busy modelling the worst case scenario in terms of what happens if we don’t put in measures and it turns out we needed them. I would hope that somewhere in the Education Department, the Department of Trade and the Treasury, there’s a team modelling what would happen if we put in a lot of measures that we didn’t need to.
However, I hope that at some point, somewhere in those Cobra meetings, someone is asking the question “and how likely is that scenario?”
— –
Postcript: I’ve been a little surprised to hear journalists and political advisors quoted as saying that they didn’t know that SAGE were asked to provide worst case scenarios because a qick Google search gives this document as the first option
Note two things. First that it is explicitly headed as “Reasonable worst case scenario planning.” This makes it overtly clear that the task of setting out the scenarios is to help Government departments plan for the possible worst case (note there may have been other even worse scenarios that SAGE could imagine but were not considered reasonable).
Secondly, it goes on to say “It should be noted that this is a scenario, not a prediction.” That the authors felt the need to spell this out in bold indicates that they were responding to misconceptions at the time. In other words Professor Medley’s distinction between scenario and prediction should not have been new news to journalists writing in December 2021 given that the question had been addressed as early as July 2020.
Faithroots 