Read on Twitter
UK headlines on Friday reported that only 20% of #COVID19 cases have any symptoms at all, quoting the Health Secretary quoting the Office of National Statistics.
This number is very wrong.
A thread on what I think happened. https://www.itv.com/news/2020-06-11/coronavirus-study-finds-as-many-as-80-of-positive-cases-do-not-show-symptoms/
This number is very wrong.
A thread on what I think happened. https://www.itv.com/news/2020-06-11/coronavirus-study-finds-as-many-as-80-of-positive-cases-do-not-show-symptoms/
The June 5 ONS report gives the prevalence, incidence, and the probability of symptoms given a positive test result in this study: P(symptoms | +) = 29%. In the report summary, no caveats with the latter result - no confidence interval or sample size.
https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/bulletins/coronaviruscovid19infectionsurveypilot/5june2020#number-of-people-in-england-who-had-covid-19
https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/bulletins/coronaviruscovid19infectionsurveypilot/5june2020#number-of-people-in-england-who-had-covid-19
Down in the details, we find that P(symptoms | +) = 29% was from just 88 people, and here large uncertainty is acknowledged: 95% confidence interval 19% to 40%.
Context is important for interpretation. It would have been helpful to mention here that many other studies find very different results for P(symptoms | covid). A meta-analysis found P(symptoms | covid) to be 85%. https://doi.org/10.1101/2020.05.10.20097543
Careful now. Notice I switched from P(symptoms | +), i.e. symptom frequency in this study's positive results, to P(symptoms | covid), i.e. symptom frequency in covid in general. The latter is what we actually want, and the value changed massively with the switch. Why?
The ONS report says "if any of these [88 individuals] are false-positives this would have a large effect on the results."
Do we expect any? Yes. Many in fact.
Do we expect any? Yes. Many in fact.
This was random community testing by design, in which only 0.41% of people tested positive. i.e. any one person's prior probability of testing positive here was P(covid) = 0.41%.
Bayes' Theorem gives the ratio of true + to false +:
P(covid | +) / P(not covid | +) =
sensitivity P(covid) / (1 - specificity) (1 - P(covid))
ONS said sensitivity between 85-95%, specificity "above 95%". I'll say 90% and 99% respectively (do try others).
P(covid | +) / P(not covid | +) =
sensitivity P(covid) / (1 - specificity) (1 - P(covid))
ONS said sensitivity between 85-95%, specificity "above 95%". I'll say 90% and 99% respectively (do try others).
Then that ratio is 0.37. Which means ~70% of the 88 individuals were false positives. Clearly, 71% of them having no symptoms does not tell us this is typical of covid!
Our best estimate from appropriate studies is that roughly 85% of people with covid will get symptoms. Not 20%.
Our best estimate from appropriate studies is that roughly 85% of people with covid will get symptoms. Not 20%.
I contacted ONS on Friday, no reply yet. I don't like jumping the gun but I didn't like waiting either - Friday's headlines may have harmed public perception of covid risk.
Warning 1: if you are told you're positive and you need to isolate, believe it! The problem above largely goes away if we already suspected someone might have the disease before testing them. In this study, we didn't: random community testing.
Warning 2: don't forget "no symptoms ever" is not "no symptoms yet". Much transmission happens before symptoms. This is why we need lockdowns or fast & effective quarantining of contacts, to stop transmission from people unknowingly spreading covid. https://twitter.com/ChrisWymant/status/1263072171833929734
Warning 3: many false positives in the ONS study does not mean covid is much less common than they say, because there are false negatives as well. They did consider the effect of inaccurate test results when estimating prevalence.
Update: In the ONS report one week later, 67% (56-77%) of 97 individuals reported never having experienced symptoms. With above assumptions, ~70% still expected to be false positives.
https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/bulletins/coronaviruscovid19infectionsurveypilot/12june2020#number-of-people-in-england-who-had-covid-19
https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/bulletins/coronaviruscovid19infectionsurveypilot/12june2020#number-of-people-in-england-who-had-covid-19
(If you're new to Bayes' Theorem note that this is situation is a classic "paradox" of probability in medicine.) https://www.scientificamerican.com/article/what-is-bayess-theorem-an/
