Read on Twitter
1/ In this thread, I show the possibility of inflated covid-19 case numbers in Victoria, based on post-market RT-PCR test kit validation published by the Australian Government.
2/ Australia ordered 10 million RT-PCR test kits from Beijing Genomics Institute (BGI), and while most of them are in reserves, only Victoria has confirmed to be using them.
https://www.theguardian.com/australia-news/2020/may/13/coronavirus-tests-brought-to-australia-by-andrew-forrest-at-200m-cost-to-taxpayers-not-being-used
https://www.theguardian.com/australia-news/2020/may/13/coronavirus-tests-brought-to-australia-by-andrew-forrest-at-200m-cost-to-taxpayers-not-being-used
3/ BGI's RT-PCR testing kits received expedited approval from the TGA.
On 10th June 2020, the Doherty Institute published a post-market validation of these kits. This is the only validation study we have available in Australia for RT-PCR tests. https://www.tga.gov.au/covid-19-testing-australia-information-health-professionals
On 10th June 2020, the Doherty Institute published a post-market validation of these kits. This is the only validation study we have available in Australia for RT-PCR tests. https://www.tga.gov.au/covid-19-testing-australia-information-health-professionals
4/ The study utilized 3 laboratories, each using different equipment for processing samples. Each lab was tested for their sensitivity and specificity from sample sizes of 156 and 115 respectively. https://www.health.gov.au/resources/publications/post-market-validation-of-the-beijing-genomics-institute-bgi-sars-cov-2-real-time-pcr-platform
5/ Sensitivity measures the proportion of +ve that are correctly identified (e.g % of sick people correctly identified as having illness)
Specificity measures the proportion of -ve that are correctly identified (e.g % of healthy people correctly identified as not having illness)
Specificity measures the proportion of -ve that are correctly identified (e.g % of healthy people correctly identified as not having illness)
6/ Another way to think about it is:
High sensitivity = smaller # of false negatives (i.e. negatives that are actually positives)
High specificity = smaller # of false positives (i.e. positives that are actually negatives)
High sensitivity = smaller # of false negatives (i.e. negatives that are actually positives)
High specificity = smaller # of false positives (i.e. positives that are actually negatives)
7/ The results show:
Lab 1 having lower sensitivity but higher specificity
Lab 3 having higher sensitivity and lower specificity
Lab 2 having results ranging in the middle for both measures
Lab 1 having lower sensitivity but higher specificity
Lab 3 having higher sensitivity and lower specificity
Lab 2 having results ranging in the middle for both measures
8/ The most ideal test has both high sensitivity + specificity, rendering the lowest possible false negatives and false positives, but this isn't always achievable. And there is often a trade-off between the two.
9/ Given the results and the conservative approach to covid-19, I do believe the VIC government would prefer to test with higher sensitivity rate to detect as much as the virus as possible, thus opting for testing equipment of Lab 3, which has the lowest specificity.
10/ But nonetheless, we see from the results that there is up to a 2.5% false positive rate (0%, 0.9%, 2.5%) across the 3 labs. As mentioned before, I believe this number tends to be closer to 2.5% than to 0%, given the strong emphasis on early detection and "test, test, test".
11/ The specificity of 97.5% per se should be considered to be acceptable, IF the population's real infection rate is high. But given a population with low prevalence of the virus, the false positive number generated could pose to be significant, and set off false alarm.
12/ If Victoria performs 30k tests per day with BGI test kits, from this one validation study alone we should expect up to 750 positive cases, in the scenario of a zero covid-prevalence population. More cases expected where virus is present.
13/ Re-testing of positive samples is possible, given sufficient nucleic acid volume, yet false negatives should currently be considered more dangerous. Re-testing 30k samples would be impractical. Each test takes a few hours to process, and the procedure is costly.
14/ For the past two months, Vic has been consistently averaging 20-30k tests per day, and the range of cases fall approx between 200 -750, somewhat in line with the false positive rate of the study.
15/ Conclusion 1: Covid-19 cases in Vic likely inflated from false positives, given that 1) Vic is using BGI kits, 2) the validation study is accurate, 3) re-testing is not performed.
16/ Conclusion 2: If covid-19 testing in Victoria truly has a significant false +ve rate (even 0.5% would be 150/30k), why is this info not given to the public? Why are we blindly trusting a single metric (i.e. covid-19 "cases") to determine the fate of the entire state?
/End
/End
