How do we measure how well the flu vaccine works every year? We use an observational study called the "test negative design." A few tweets on how it works, and why it will be a big part of ongoing COVID-19 vaccine evaluation. 1/8

Figure source: Fukushima et al. (2017) Vaccine

In the test negative design, individuals with disease symptoms seek healthcare and testing. If they test positive, they are TEST POSITIVE CASES. If they test negative (and their symptoms are caused by something else), they are TEST NEGATIVE CONTROLS. 2/8

We can then look back and see how many of the test positive cases were vaccinated, and how many of the test negative controls were unvaccinated. Where the vaccine works well, there won't be many vaccinated people testing positive. 3/8

On the other hand, a flu vaccine doesn't protect against other respiratory viruses, so we would expect a mix of vaccinated and unvaccinated controls, reflecting vaccination coverage in the underlying population. We can estimate vaccine effectiveness as 1 minus the odds ratio. 4/8

Test negative designs are valuable because they are inexpensive. They can be integrated into existing surveillance systems. They are far cheaper than a cohort study where you have to follow many thousands of people to observe a small number of cases. 5/8

They also reduce a certain type of bias that can occur because the type of people who go to their doctor when they feel sick may also be the type of people who get vaccinated. But this study restricts to these "care seeking" individuals (and those who can access testing). 6/8

This will be a vital study design for evaluating COVID-19 vaccines. While randomized trials are the gold standard, they are increasingly difficult to do, as they may be unethical or people may not consent to participate. Observational study designs will fill in the gaps. 7/8

So it is critical that these study designs are of high quality, because we will use them to determine how durable vaccines are, how well different vaccines work against new variants, and so on. I hope this primer helps! 8/8 https://science.sciencemag.org/content/370/6518/763.summary