Read on Twitter
A serious reasoning error that is particularly common among educated people is to argue that if a study hasn't been done on a particular question we have 'no data', and therefore no basis on which to form beliefs or act.
This is incorrect and dangerous. 1/
This is incorrect and dangerous. 1/
There is almost always a reference class we can use to make a sensible prediction, and often we can reason about the causal mechanism directly.
To see how this is clearly wrong, consider these two extreme cases: 2/
To see how this is clearly wrong, consider these two extreme cases: 2/
a. Before a vaccine trial is conducted we have 'no data' on how likely it is to work.
But we can estimate the likelihood just fine! To start with we can just check the base rate of all vaccine trials.
What fraction of all vaccine trials in history have had +ve results? 3/
But we can estimate the likelihood just fine! To start with we can just check the base rate of all vaccine trials.
What fraction of all vaccine trials in history have had +ve results? 3/
That gives a much better estimate than pure agnosticism.
b. Imagine now that the vaccine trial results have come back positive. But by chance all the vaccines were given in a blue room.
So, will the vaccine still work if given in a red room? I guess we have 'no data'.
4/
b. Imagine now that the vaccine trial results have come back positive. But by chance all the vaccines were given in a blue room.
So, will the vaccine still work if given in a red room? I guess we have 'no data'.
4/
So can we say nothing about about whether vaccines work in red rooms?
That is plainly ridiculous — we understand the causal mechanism and it doesn't involve room colour, so in fact we can give a near definitive answer to that question. 5/
That is plainly ridiculous — we understand the causal mechanism and it doesn't involve room colour, so in fact we can give a near definitive answer to that question. 5/
I think this confusion is encouraged by people who study a bit of science but never study philosophy of science, so end up with a garbled epistemology — one more peculiar and foolish than whatever they started with. 6/
It is also encouraged by bureaucratized thinking, where the key is to avoid responsibility by never making personal 'judgement calls', and instead robotically following a cover-your-ass process
The error does most damage when we need to move fast in an uncertain environment. 7/
The error does most damage when we need to move fast in an uncertain environment. 7/
I've been thinking about it this week because I keep reading people on Twitter saying we have 'no data' and no ability to say whether COVID vaccines work after a single dose, because a trial studying a single dose hasn't been conducted.
What complete bollocks! 8/
What complete bollocks! 8/
To start with the Pfizer vaccine shows new infections crashing a week before patients received the second dose.
But even if they didn't, we could look at the base rate among all vaccines — what fraction of all vaccines offer protection for ~6 months after the first dose? 9/
But even if they didn't, we could look at the base rate among all vaccines — what fraction of all vaccines offer protection for ~6 months after the first dose? 9/
More than half? Less than half? That is super informative!
Furthermore, we know how vaccines work, and people who are experts in immune reactions probably have a trained intuition on how likely one is to get immunity after a first exposure to an antigen. 10/
Furthermore, we know how vaccines work, and people who are experts in immune reactions probably have a trained intuition on how likely one is to get immunity after a first exposure to an antigen. 10/
They'll also have a sense of how strong human immune reactions are to COVID surface proteins as compared to those on other viruses.
If they think one shot will work with 70% probability or 30% probability, that's a big update either way. 11/
If they think one shot will work with 70% probability or 30% probability, that's a big update either way. 11/
Where something hasn't been studied extensively, the resulting probabilities are likely to fall between 20-80% rather than near 0% or 100%.
But that's normal and fine, and they can go in an expected value calculation nonetheless. 12/
But that's normal and fine, and they can go in an expected value calculation nonetheless. 12/
(The folks above are separately making an error where they neglect risks in the status quo.
They call giving one dose 'risky' but if we were already planning to give one dose we could equally say two doses is wildly risky — because you only vaccinate half as many people!) 13/
They call giving one dose 'risky' but if we were already planning to give one dose we could equally say two doses is wildly risky — because you only vaccinate half as many people!) 13/
All estimates are subjective probabilities.
No two cases are exactly the same, and we always have to reason from nearby analogies — which means we always have relevant information we can use to guide a decision. 14/
No two cases are exactly the same, and we always have to reason from nearby analogies — which means we always have relevant information we can use to guide a decision. 14/
On top of that, inaction or 'going with the default' is as much an action as choosing the alternative!
Either way you're making an implicit estimate about how good the alternative is. 15/
Either way you're making an implicit estimate about how good the alternative is. 15/
If you go with the default you're not actually remaining agnostic until the 'data' come in — you're actively deciding that the alternative has a lower expected value.
So, don't let people get away with dodging the responsibility to choose using confused philosophy. 16/
So, don't let people get away with dodging the responsibility to choose using confused philosophy. 16/
Added: Some people are reading the above as an argument for the UK's one-dose-first vac approach.
As it happens I do think the UK is making the right call but nothing I've said above demonstrates that. I'm just saying you can and must decide using all the available evidence. a/
As it happens I do think the UK is making the right call but nothing I've said above demonstrates that. I'm just saying you can and must decide using all the available evidence. a/
To support the UK's approach we'd look at all the evidence and conclude the prob of one dose working for 3-6 months is >50%, either 'cos this is true of most vaccines or we have reason to think it's true of this one.
Also need to address concerns around evolved immunity, etc. b/
Also need to address concerns around evolved immunity, etc. b/
