Read on Twitter
Some rambling thoughts on COVID-19 modelling.
Since COVID-19 began, I've seen a few different *types* of models being used to make forecasts. SEIR (susceptible/exposed/infected/recovered) models have been used quite extensively for official forecasting purposes.
1/N.
Since COVID-19 began, I've seen a few different *types* of models being used to make forecasts. SEIR (susceptible/exposed/infected/recovered) models have been used quite extensively for official forecasting purposes.
1/N.
Other models have also been applied: time-series models by @spyrosmakrid and @fotpetr: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0231236
Also, models based on the Gompertz function have been applied and seem to be quite successful. This has mainly been applied as a type of growth curve analysis.
2/N
Also, models based on the Gompertz function have been applied and seem to be quite successful. This has mainly been applied as a type of growth curve analysis.
2/N
In what way are these models similar/different? All three types can be used to produce forecasts. SEIR-type models rely on a set of assumptions regarding infection/recovery/fatality rates which can be difficult to infer from the data available...
3/N
3/N
...unless one is following a very careful calibration program, like the https://covid19-projections.com/ of @youyanggu. At the outset of an epidemic, though, one can set these parameters with suitable expert judgement. Model results produced in this way seem to be very useful
4/N
4/N
for explaining what the consequences of an epidemic might be through scenarios. As an outsider who doesn't use SEIR-type models often, it seems hard to make accurate forecasts, since errors get exponentiated, see from @nntaleb, @yaneerbaryam and @DrCirillo:
5/N
5/N
https://forecasters.org/wp-content/uploads/Talebetal_25062020.pdf.
On the other hand, time-series or growth curve analysis fits models that extrapolate observed experience. If one doesn't have data yet (like at the outset of an epidemic) or reliable data these models are not useful.
6/N
On the other hand, time-series or growth curve analysis fits models that extrapolate observed experience. If one doesn't have data yet (like at the outset of an epidemic) or reliable data these models are not useful.
6/N
This reminds me very much about models actuaries apply for reserving for general insurers. At the outset of reserving for a cohort of policies, one has little information on which to apply extrapolative models, thus one usually relies on judgement. As information...
7/N
7/N
trickles in and the experience becomes more stable, one usually moves to extrapolative models. This can take a long time (years) if the data is volatile and if the situation is changing rapidly.
What does this mean for forecasting COVID-19 in SA?
8/N
What does this mean for forecasting COVID-19 in SA?
8/N
South Africa's demographic data are probably among the best in Africa, but are not "perfect". Deaths are not completely reported, and population data can be skewed by misreporting:
http://ronaldrichman.co.za/wp-content/uploads/2017/12/Thesis.pdf
9/N
http://ronaldrichman.co.za/wp-content/uploads/2017/12/Thesis.pdf
9/N
It is hard to know exactly how these issues that existed before COVID-19 affect the death reports we get. On the other hand, even in developed countries, there are deaths in excess of those reported as being due to COVID-19 being analyzed:
https://www.ft.com/content/a26fbf7e-48f8-11ea-aeb3-955839e06441
10/N
https://www.ft.com/content/a26fbf7e-48f8-11ea-aeb3-955839e06441
10/N
I thus view the MRC's report on excess deaths in SA as the key data point to track to understand the combined impact of COVID-19 and related deaths in SA:
https://www.samrc.ac.za/sites/default/files/files/2020-07-22/WeeklyDeaths14July2020_0.pdf
11/N
https://www.samrc.ac.za/sites/default/files/files/2020-07-22/WeeklyDeaths14July2020_0.pdf
11/N
Of course, this report cannot tell us too much about the proportion of the deaths are to COVID-19 "directly". That will take a much longer time to estimate, if it is possible. See here for a view on the number of AIDS deaths in SA:
https://journals.lww.com/aidsonline/Fulltext/2016/03130/HIV_AIDS_in_South_Africa__how_many_people_died.15.aspx
12/N
https://journals.lww.com/aidsonline/Fulltext/2016/03130/HIV_AIDS_in_South_Africa__how_many_people_died.15.aspx
12/N
If the underlying data are not capturing all of the deaths due to COVID-19, I would be wary of fitting any extrapolative models to SA data, unless some allowance is made for the unreported deaths. It would be valuable to correct the underlying data and then extrapolate.
13/N
13/N
Until this can be done successfully, for providing information for planning and other purposes, models of the SEIR-type seem like our best bet. Some nuanced experiences from experts using these models are written up here:
https://www.news24.com/news24/columnists/guestcolumn/the-sa-covid-19-modelling-team-modelling-a-pandemic-on-scarce-data-and-unknowns-20200723
14/N
https://www.news24.com/news24/columnists/guestcolumn/the-sa-covid-19-modelling-team-modelling-a-pandemic-on-scarce-data-and-unknowns-20200723
14/N
In the end, models can only tell us part of what we need to know and understanding the properties of the process we are dealing with is probably more important for decision making than single point forecasts.
15/N
15/N
From @nntaleb and his colleagues:
"Sufficient –and solid – evidence, in particular for risk management purposes, is already available
in the tail properties themselves. An existential risk needs to be killed in the egg, when it is still cheap to do so."
16/N
"Sufficient –and solid – evidence, in particular for risk management purposes, is already available
in the tail properties themselves. An existential risk needs to be killed in the egg, when it is still cheap to do so."
16/N
"Secondly, unreliable data–or any source of serious
uncertainty–should, under some conditions, make us follow the "paranoid" route."
17/N
TBC
uncertainty–should, under some conditions, make us follow the "paranoid" route."
17/N
TBC
On "killing in the egg": My views early on were that SA should have taken immediate action to control the spread of COVID-19:
https://twitter.com/RichmanRonald/status/1239097049867485184?s=20
And at the outset of lockdown I worried if it was too late:
https://twitter.com/RichmanRonald/status/1242182306217046016?s=20
18/N
https://twitter.com/RichmanRonald/status/1239097049867485184?s=20
And at the outset of lockdown I worried if it was too late:
https://twitter.com/RichmanRonald/status/1242182306217046016?s=20
18/N
On "unreliable data": I would be keen to see someone come up with a methodology for allowing for the uncertainty in the SA data and propagating that uncertainty to an extrapolative model. The resulting estimates and confidence bands would be quite informative.
19/19
19/19
