Ok at the request of @PerPersvensson let's do a
Bayes Analysis of the HERS trial circa JAMA 1999

Estrogen/progesterone vs Placebo for reduction of MI/CV death. https://twitter.com/PerPersvensson/status/1355426831797772290

Here are the events to put in the 2x2 table and the published CI and RR

First just look at the data. No diff in outcomes and wide CI from 20% benefit to 22% harm.

Going to be hard for prior beliefs to budge our posterior

Posterior is jargon for after seeing the data.

Goodman has a nice paper on using Bayes https://courses.botany.wisc.edu/botany_940/06EvidEvol/papers/goodman2.pdf

First step is to calculate a likelihood ratio. We do this in our heads all the time as doctors.

The LR is called the Bayes Factor and it is simply the ratio of the probability of the null H over the alternative H

Now we have to think about how extreme the data is.
At UConn med school, we were graded with Z scores--which are a measure of how many standard deviations you are from the mean in standard Gaussian bell curve.
Z of 2 was quite good. Z of 0 was average. Negative Zs were bad.

There are many P value to Z score calculators online https://www.gigacalculator.com/calculators/p-value-to-z-score-calculator.php

Here a p value of 0.02 yields a really high Z score --which makes sense.

This step is tricky and I don't quite exactly understand the math of why a minimum BF (likelihood ratio) is equal to
e raised to the negative Z squared/2
But I don't think it's that crucial, BF is just a ratio

Back to Bayes Theorem that states how you interpret data depends on the likelihood ratio (evidence) x your prior belief.

Positive stress test means a lot more in a smoker with typical cp than a young person with stabbing pain lasting seconds.

Now let's use BF and prior beliefs to interpret the HERS trial.

First the BF is very high (1 is highest). This makes sense b/c the P value was high. Or data was not extreme.

Sometimes BF are cited as inverse.

If you don't have a strong belief before a trial, you are 50-50 on the null being true, the odds are 1
If you are skeptical, you think say 70% chance of the null, then the odds are 70/30 or 2.3
If you are enthusiastic, only a 25% chance of null, the odds are 0.33

Look what happens in HERS

Even if you are enthusiastic, there is still a 1 in 4 chance that the null hypothesis is true and there is no effect.

That's not persuasive data.

But wait, Bayesian analysis gets better!!
As docs, we don't care so much about ANY benefit, @raj_mehta says we want to know the minimally clinically important difference. Say 20% (statins are ≈ 25%)

Bayes can do this, but you need help of worksheet or software that can plot probability density functions

Note this is not DIY. Many papers have been published on this.

Now to HERS. Let's start with a non-informative prior belief.
You enter a mean of 0 and SD of 2(large)

You can see the prob of a 20% benefit is only 4%.
You can also see the probability of 10% harm is > 1 in 10.

@PerPersvensson wants us to be enthusiastic. Ok.
Now you enter the CI you think it might be. Say 25% reduction to 1% reduction. (0.75-0.99)

This is where prior data could play. Like say for higher death rates in EXCEL, you could enter NOBLE death rates (no diff)

Look what happens to HERS with an enthusiastic prior
Blue is data--hovering around null
Yellow is your enthusiastic prior
Red is the posterior or Blue (data) x Yellow (Prior)
Red is a tighter curve but still only a small chance of benefit
Now, though, less chance of harm.

Summary -even with enthusiastic prior beliefs, HERS does not persuade you of a clinically important benefit.

I love this b/c we can tell patients the probability of different degrees of benefits over different prior beliefs.

How did I do @kaulcsmc @brophyj @djc795 @dailyzad

Comments welcome -- I am learning the guardrails here.

Let me know.