(1/n) A few weeks ago I noted that if you expecting vaccine efficacies in the 30-70% range, knowing an interim says “continue” doesn’t say much. With Pfizer’s very optimistic 90%, should we have a more optimistic prior on Moderna? And if so how negative would “continue” be? https://twitter.com/CNBC/status/1326636739117592584

(2/n) Here is the old thread for reference, where I used priors that seemed to be in line with an expectation of more modest effectiveness. Pfizer’s 90% is above that range. https://twitter.com/KertViele/status/1307463136736354308

(3/n) Moderna is using a hazard ratio based analysis (Cox Model). For simplicity and speed, I’m going to report these based on event splits (events are the primary input to the Cox Model). There are a number of unknowns.

(4/n) We don’t know the number of Moderna events (just “substantially more” than 53). Moderna was going to require p<0.0002 at 53 events, or p<0.0073 at 106 events (max events 151). Somewhere (SAP) they should have a plan if it isn’t either of these. We don't know that plan.

(5/n) For simplicity, I’m going to assume they have 106 events (the original second interim, close to Pfizer's 94) and need p<0.0073 to declare efficacy. If they have 75 events or so, then unclear how they are handling it (I’ll update this if someone finds this out).

(6/n) Suppose we use the posterior distribution from Pfizer as the prior for Moderna, so our prior beliefs are summarized in the left panel (very optimistic the vaccine is highly effective). If Moderna says “continue”, the right panel gives the updated beliefs about Moderna.

(7/n) This would be major news, unlike many interims where a realistic prior doesn’t change much based on an interim “continue”. We would significantly lower our beliefs about Moderna’s efficacy (but still be optimistic!).

(8/n) It would also completely beg the question of how our beliefs about Pfizer should be adjusted (they should move toward Moderna, but by how much?)

(9/n) Another prior, little more pessimistic and variable for Moderna, and the resulting posterior after a “continue”. Still pretty major shifts

(10/n) Note with either of these priors we would predict a >99% chance of a successful interim. If the true VE is over 80%, you are just very likely to declare success. If that doesn't happen, we would have to re-evaluate our beliefs significantly.

(11/11) People often overreact to interim “continue” results. This MIGHT be an exception. If we are now pretty optimistic about Moderna, an interim “continue” meaningfully reduces our optimistic belief. But again, all the CIs remain in the “good” range.