Beware of using experiments to support your argument

Photo by Polina Tankilevitch on

I am a firm believing in the helpfulness of both empirical evidence and forecasting models to help us make decisions. However, during COVID19, we seem to have run into problems whenever results are shared and there hasn’t been great care to understand what they are meant to be telling us. So, for example we get a lot of panic reactions to data and models that suggest worst case scenarios. However, reports have also been shared as “good news” stories when I’m not sure they prove as much as they claim either.

For example, lockdown sceptics have been using this report to show that there is no need to worry about the risk of children heading of to school and bringing the virus home. If we can prove that families with children are at no greater risk than those without then surely that’s a positive thing.

But does the report actually prove that? Note the following facts about the report:

  1. The study was carried out between February and August missing out the crucial period from September to November when children were back in school.
  2. It focuses on adults aged >65 and distinguishes between those who have children at home and those who do not.
  3. It concludes that adults under 65 with children under 12 at home are at a reduced risk from COVID-19. There is also a small risk of increased infection for those with children aged 12-18 at home.

Now, let me make a few observations about this.  First of all, the primary concern in society has not been that parents living with children would pick up coronavirus and become seriously ill but rather that families would pass on the virus to vulnerable elderly relatives.  Secondly, the adult age range here is quite broad.  Under 65 means a span for 47 years from 18-65.  This means that you will get different age categories within the main band. 

Now, in general, it is likely that someone with a  primary aged child is going to be under 40 and at the younger end under 35 or even 30.  Even with teenage children it is likely that most adults with youngsters at home will be in their 40s at most. Meanwhile, those without children at home are more likely to have not had children due to health concerns or already be into their 50s and sixties.  In  other words, those without children are more likely to be in an increased vulnerability category.

The report is too broad in its conclusions and fails to distinguish cause from correlation. It is not yet peer reviewed and yet it is already being waved around as the great hope of those wanting a speedy end to lockdown. The result is that in my opinion, it fails to offer genuinely helpful information and advice.

It is so important that we learn to observe what models, experiments and studies are telling us before rushing to use a report to back our particular pet idea or conspiracy theory.  This is important whichever side of the debate we find ourselves on.

Leave a comment

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s