I want to follow up on a thread I started but abandoned last night. I have been working on clinical prediction rules that predict death in folks with identified covid (obviously the tip of an iceberg). It is remarkably easy to predict those at increased risk of death.

The idea with a prediction rule is that you take a (usually) logistic regression model (something that has a yes=no outcome) and you convert the model coefficients into a simple point score that folks can use to evaluate risk. These rules have ...

a bunch of applications...can use them to identify people who should be hospitalized (e.g., with pneumonia, like here: https://www.nejm.org/doi/full/10.1056/NEJM199701233360402) vs. treated safely at home.

identify folks who you want to include in clinical trials, or (in the context of opening up) identify those who may be at particular risk if exposed to COVID-19

So here is the rule as tweeted last night: https://twitter.com/DFisman/status/1273042171227144192?s=20

It's actually one of four rules but it works best and is parsimonious I think.

It uses something called "split halves" validation, where we have so many covid cases in Ontario now that we can split the dataset in two. One set is used to construct (derive) the rule, and then...

it is tested in the other half (validation set). This is what the score looks like (line) vs. actual probability of death (circles). The size of the circles is proportionate to deaths at that score range.

It's pretty good, and "well calibrated"...as the score goes up actual risk of death goes up. The second feature of prediction rules is discrimination, how well they divide people into baskets. We do this using "ROC curves" (derived from WW II studies of radar).

We can't do an ROC tweetorial right now but the area under these curves is an index of discriminatory ability across all positive cut-points. AUC = 0.5 would be a coin toss.

The AUC for this model is > 0.9 in both derivation and validation sets, which is really good.