Read on Twitter
1/ In support of well-conducted trials, & proper scientific conclusions: A thread. Special Thanks to Andre Kalil, MD at University of Nebraska Medical Center, for content creation & encouragement. I remain your #MedTwitter minion. Disclosure: No financial COI. I’m an ACTT site PI
2/ A mentor frequently told me “Nature is always right & you are always wrong”. Assumptions re: mechanism, biologic plausibility, can’t invalidate actual observations. Parsimony insists we reach scientific conclusions explaining incongruent results, not just pick the one we like
3/ Many questions about Remdesivir followed publication of SOLIDARITY & its varied observations from ACTT. We have been asked many questions & are sure you have as well. Importantly, our patients deserve the highest level of science.
4/ Some say Remdesivir is a panacea. Some are convinced that it does nothing…These two opposite extreme conclusions are a debasement of #science. ACTT-1 showed a clear benefit & SOLIDARITY did not show any impact. Can we hold these 2 results in the same hand?
5/ There is no doubt Remdesivir provides meaningful clinical benefits when used in appropriate patient populations, based on the consistent ACTT-1 survival estimates. SOLIDARITY pragmatic study design cannot invalidate a high quality trial like ACTT-1, it can only offer context.
6/ SOLIDARITY & ACTT observations differ. ACTT showed Remdesivir benefit of shorter time to recovery 5 days for all, & 7 days in the sickest pts & lower mortality. SOLIDARITY showed no significant improvement in time to recovery or mortality. How do we reconcile this incongruity?
7/ ACTT was a double blind placebo controlled trial, w/ rigorous data definitions for outcomes, & adverse events & strict inclusion/exclusion criteria. SOLIDARITY was a bigger “pragmatic, real world” trial. A larger n doesn’t obviate design flaws, but rather magnifies them!
8/ Efficacy (works in best circumstance) & Effectiveness (works in real world) is vital to evaluate therapy, particularly in applying research to LMIC w/ resource/personnel/infrastructure limitation. This doesn’t excuse folks from rigorous pragmatic trial design, it demands it.
9a/ SOLIDARITY had No placebo (to prevent treatment assessment bias for both efficacy & safety) No double-blinding (to prevent information bias, treatment assessment bias, adherence bias, follow up bias) No true randomization! (to prevent selection bias see next several tweets!)
9b/ SOLIDARITY PI’s could reject randomization if assigned therapy was not acceptable to the PI, or if therapy was not available. This randomization process can create serious selection bias, undermining the goal of randomization! Factors affecting PI use of this option…
9c/ When SOLIDARITY was underway, only one drug had demonstrated clinical benefits: remdesivir. Many PIs anxious to have remdesivir. W/o stratification by site or disease severity, very possible that patients who received remdesivir were sicker at baseline than controls
10/ In case you wondered about that..SOLIDARITY did not stratify enrollment by disease severity or hospital site (needed to prevent selection bias); please see tweets (7,8,9) Hospital sites were not assessed to determine a site bias…
11/ Inappropriate control selection (control were usual care from 400 hospitals in 30 countries. Treated & control pts were not comparable since local standard of care likely varies greatly in Iran, Phillipines, Peru, Indonesia, four highest enrollers of Remdesivir in SOLIDARITY
12/ But wait..There’s more. SOLIDARITY had No local data monitoring (needed for the appropriate trial conduct, data accuracy, and patient safety) & No diagnostic verification of infection (needed to ensure patients have equal proportion of confirmed diagnosis between both arms)
13/ No symptom duration (ensure tx is at equiv point in nat hx of dz)Unknown baseline severity (ensure = prognosis between arms): a pt could have COVID19 for 3d/3wks/3mo w/multi-organ failure or not...Does timing & patient severity matter? This trial design simply cannot tell us!
14a/ Supportive care & health care capacity of enrolling sites both Unknown (to ensure equal cointerventions between arms & sites) a clear determinant of outcome in COVID19. These two issues with SOLIDARITY are key to how ACTT & SOLIDARITY showed different results
14b/ Let’s say you are at a hospital with overwhelmed capacity, and limited supportive care expertise. You have a pt with pneumonia, severe sepsis & ARDS due to S aureus sensitive to all antibiotics. Which antibiotic would work? Probably doesn’t even matter…
14c/ No matter how effective the antibiotic, this patient would not survive without the needed supportive care for sepsis & ARDS which are affected both by expertise & resource limitation. Same rationale applies to COVID-19. Census and supportive care are key!
15/Missing data (to prevent information bias) Even mortality data during hospitalization were missing in final SOLIDARITY analysis. Many tout mortality as the best endpoint, except when the pt is lost to follow-up. How bout recovery time as an endpoint? No difference there right?
16a/ In SOLIDARITY, Remdesivir required 10 days of IV administration in the hospital, even if the patient was well enough to leave. If you were ready to leave at day 5, you stayed to day 10 to finish therapy. This clearly affects time to recovery when defined as “discharge”
16b/ this is not trivial as potential confounder: the longer someone needs to stay in the hospital, the higher the risk of developing nosocomial infections, thrombotic events, hospital drug administration errors, all of which are associated with higher risk of death.
17a/ The highest level of scientific rigor is a randomized double-blind placebo-controlled study with systematic clinical & laboratory data collection, clear outcomes, data & safety monitoring committee –fully achieved by ACTT-1
17b/ ACTT-1 demonstrated significant clinical benefits with remdesivir:
50% faster clinical status improvement 8 days less receiving oxygen
30% lower progression to non-invasive ventilation
44% lower progression to invasive mechanical ventilation
50% faster clinical status improvement 8 days less receiving oxygen
30% lower progression to non-invasive ventilation
44% lower progression to invasive mechanical ventilation
17c/ Despite not being powered for mortality, ACTT analysis showed 45% lower overall mortality reduction at day 14 (0.55[0.36-0.83]P=0.004)
70% mortality reduction in patients who were requiring oxygen at day 14 (0.28[0.12-0.66]P=0.002) and at day 28 (0.30[0.14-0.64]P=0.001)
70% mortality reduction in patients who were requiring oxygen at day 14 (0.28[0.12-0.66]P=0.002) and at day 28 (0.30[0.14-0.64]P=0.001)
18/ Why the discordant results? Internal Validity & Generalizability. SOLIDARITY’s low internal validity doesn’t generalize broadly enough to invalidate ACTT. ACTT high internal validity doesn’t generalize broadly enough to be applied to SOLIDARITY’s enrollment.
19/ Picking one that you like and ignoring the other based on its abstract just isn’t how science works. Heterogeneity in onset/severity/site/treatment initiation are far more likely explanations for lack of effect than “I guess that other trial was all just random chance”.
20/ Remdesivir isn’t perfect, but definitely works in the right patient/place/time. ACTT and SOLIDARITY both inform & illuminate: If you are willing to get your science on, & understand them. You can’t apply treatment without the science. Your patient’s life depends on it.
