Read on Twitter
Thread to discuss our new RCT (ELAIA-1) appearing in @bmj_latest that showed some unexpected results in the world of electronic alerts for acute kidney injury. https://www.bmj.com/content/372/bmj.m4786
We know that AKI goes unrecognized and, in theory, undertreated from large retrospective studies like this one from @dmoledina: https://www.sciencedirect.com/science/article/pii/S0272638620310441?via%3Dihub
Acknowledging that reality, many health systems (including the @NHSuk, have institute automated "alerts" for AKI).
But do they actually improve outcomes? We conducted a randomized trial to find out.
We built an automated alert into our electronic health record that was shared across 6 hospitals. It gave information about creatinine values, a link to an AKI order set, and a link to the study website.
The alert fired whenever the chart was opened while the patient had AKI until it was acknowledged by the provider. It could be seen by anyone who could enter orders (MDs, DOs, NPs, PAs - not med students, nurses, pharmacists).
Since this was an RCT - the alert only fired on 50% of patients. Building that logic into the EHR was not trivial, but that's a story for another day!
We enrolled 6,030 hospitalized adults with AKI.
The primary outcome was a composite of progression of AKI (to a higher stage), dialysis, or death within 14 days of randomization. But we looked at "process outcomes" along the way.
The alerts did move the needle a bit on some provider behaviors. They led to more IV fluids for example, and more documentation of AKI.
Overall, though, the primary outcome was similar in the two groups.
21.3% of the alert group and 20.9% of the usual care group had progression of AKI, dialysis, or death (p=0.67).
But then things got weird.
21.3% of the alert group and 20.9% of the usual care group had progression of AKI, dialysis, or death (p=0.67).
But then things got weird.
We had prespecified ( https://bmjopen.bmj.com/content/9/5/e025117) that we would look for heterogeneity of alert effect across our 6 study hospitals, and indeed we found it.
Specifically, the effect of alerting was markedly WORSE in the 2 non-teaching hospitals.
In the non-teaching hospitals, there were significantly more deaths in the alert compared to the usual care group - 15.6% vs 8.6%, p=0.003.
This was obviously very concerning to us, and after carefully checking our code, we conducted a series of post hoc mediation analyses to figure out what was going on.
I suspected it might be overly aggressive fluid resuscitation, but that didn't explain the effect.
Nor did use (or nonuse of contrast), kidney consults, or anything else we could throw at it.
Nor did use (or nonuse of contrast), kidney consults, or anything else we could throw at it.
We did a full, case-by-case review of each death in those hospitals looking for a pattern. Aside from the number being higher, we didn't find anything. Deaths happened for reasons deaths happen in the hospital (sepsis, cancer, heart failure, etc).
We reported the result to the IRB and the @NIH (who funded the study). It put much of our studies on hold while we finished an investigation.
I don't have a satisfying answer, though I wrote about our thinking here. https://blogs.bmj.com/bmj/2021/01/18/the-challenge-of-minimal-risk-in-e-alert-trials/
Given this data, I would argue that AKI alerts that do not provide patient-specific recommendations may not be useful and may engender harms that we don't fully understand.
This is why RCTs of even "common sense" interventions are critical. You may be surprised.
This is why RCTs of even "common sense" interventions are critical. You may be surprised.
What's next for us? We're already running our trial (ELAIA-2) that specifically makes recommendations about certain drug therapies in AKI.
If YOU want to explore the ELAIA-1 data you can - a deidentified dataset is posted free for all here:
https://datadryad.org/stash/dataset/doi:10.5061/dryad.4f4qrfj95
No you don't need my permission to download. Have at it! #OpenData is awesome. Publicly funded data should be free.
https://datadryad.org/stash/dataset/doi:10.5061/dryad.4f4qrfj95
No you don't need my permission to download. Have at it! #OpenData is awesome. Publicly funded data should be free.
We'll be discussing this on #nephjc Tuesday the 26th at 9pm EST. Join us then! (or ask questions here).
Thanks to so many people for working on this including Melissa Martin @SimonovSays @dmoledina @KidneydrChirag @renalgal @JasonHenryG @adityaxbiswas @tanima_arora @tizucal Caitlin Partridge @EricaAnnMoreira @StephenLatham @YaleScleroderma
@YaleIMed @Yale @YaleNephFellows
@YaleIMed @Yale @YaleNephFellows
Also thanks to our external advisors Harv Feldman, Amit Garg and @PaulPalevsky.
