The Johnson & Johnson Vaccine: A deeper dive into the available data
Disclaimer: The information included in this post is not medical advice. If you have concerns regarding the COVID-19 vaccine, consult your doctor. The following analysis is intended as a case study for how one can use data science to gain a more complete understanding of public health topics in the news (including those pertaining to the COVID-19 pandemic).
As of this writing in May 2021, vaccines produced by three different pharmaceutical companies (Moderna, Pfizer, and Johnson & Johnson) have been approved for use in the United States. The first two approved (those manufactured by Moderna and Pfizer) share several similarities — they both require two doses and they both use genetic material called messenger RNA (or mRNA for short) to induce immunity in vaccine recipients. When they were authorized for emergency use in the US in December 2020, they became the first mRNA vaccines to be approved for use on a wide scale. This was a significant milestone — and over the past few months, the administration of these vaccines has widely been considered a success in terms of their impact on the fight against COVID-19.
The other vaccine (produced by Johnson & Johnson) differs from the other members of the group in a few ways. First, it only requires one dose. This provides an advantage over the other two as they require recipients to receive two shots several weeks apart. Second, the Johnson & Johnson vaccine uses an inactivated virus to deliver viral DNA (as opposed to mRNA) to induce immunity. Unlike the method used by the other vaccines, this type of vaccine has been used before (e.g. Hepatitis B). However, despite these apparent advantages of this option, since it received emergency use authorization in February 2021, the J&J vaccine has been met with some criticisms regarding its efficacy and safety. This has led to public concerns over this vaccine option, leading it to even receive the SNL treatment (skip to 2:08 for the reference in the sketch):
This blog post will explore two of the reasons why some believe that the Johnson & Johnson vaccine might not be“totally safe” (as portrayed in the SNL sketch). The first is that the reported efficacy of the J&J option is lower than that of the Moderna and Pfizer vaccines. However, as is often the case, a deeper dive into the data can provide some more information here. The second concern that people have with the J&J vaccine is that it has been associated with a potentially dangerous side effect. After reviewing the data, the CDC/FDA determined that the “known and potential benefits outweigh (the) known and potential risks”. This post will look at a simple way to assess the main risks/benefits referred to by the CDC/FDA using publicly available data.
While the exact numbers vary depending on source, the reported efficacy numbers for the three vaccines are typically around:
- Pfizer is 96% effective
- Moderna is 95% effective
- Johnson & Johnson is 66% effective
Source: https://peterattiamd.com/pauloffit2/
On face value, a difference of ~30% seems significant. However, it is important to ask the question — what are these efficacy rates even measuring? In order to demonstrate how these numbers are calculated and to define some important terms, we can use a hypothetical case as an example…
Tosus is a (hypothetical) disease that causes flu-like symptoms. My (hypothetical) pharmaceutical company, HodgeCo, is testing a vaccine for Tosus called Pro-VAX. In our clinical trial, patients are randomly assigned to two groups — the treatment group (the ones who receive Pro-Vax) and the control group (the ones who receive a placebo). Control groups are used for comparison in order to measure the effectiveness of the experimental vaccine (i.e. if the treatment group shows better results that are statistically significant, we can infer that the vaccine played a role in reducing the rate of disease in that group). One way that scientific studies measure the effectiveness of a vaccine (or other drug) is in terms of “risk reduction” (i.e. the difference in risk that a patient who receives a vaccine is exposed to as opposed to someone who does not receive it).
Here’s some made up data for my theoretical clinical trial:
There are two ways to measure risk reduction:
- Absolute risk reduction: This is the difference disease rates between the two groups (0.741% — 0.111% = 0.63%)
- Relative risk reduction : This is the absolute risk reduction divided by rate of disease in control group ((0.741% — 0.111%)/0.741% = 85%)
Source: https://peterattiamd.com/ns001/
Now that we know a little about the difference between absolute and relative risk reductions, let’s return to our COVID-19 efficacy rates.
- Pfizer = 96%
- Moderna = 95%
- Johnson and Johnson = 66%
These actually refer to the relative risk reductions observed during the clinical trials for each of these vaccines. Here are the absolute risk reductions:
- J&J: ~1.7%
- Moderna: ~1.2%
- Pfizer: <1%
Source: https://peterattiamd.com/pauloffit2/
Two things stand out to me — 1) these numbers are much closer together than the relative risk reductions, which leads us to infer that these vaccines provide a similar level of protection against COVID-19; and, 2) the order of the vaccines are actually inverted (when comparing relative to absolute risk reductions) — Johnson & Johnson has the greatest absolute risk reduction, while Pfizer has the lowest. A possible explanation for this is that the Pfizer trial could have had healthier participants overall (and therefore, less cases of COVID-19 in both the control and treatment groups), while the J&J trial could had participants who were at greater risk for COVID-19 (regardless of treatment group).
The other critique of the J&J vaccine that has made headlines is around concerns over cerebrovascular sinus thrombosis (clots in veins that drain blood from the brain). On April 13, 2021, the US Center for Disease Control and Prevention (CDC) and Food and Drug Administration (FDA) recommended a pause to the administration of the Johnson & Johnson vaccine following reports of a possible link to the blood clots. However, after a more thorough review of the data, the CDC and FDA elected to resume use of the J&J vaccine in the US. Here is an excerpt of the announcement from the Center for Disease Control and Prevention website:
Source: https://www.cdc.gov/coronavirus/2019-ncov/vaccines/safety/JJUpdate.html
The CDC announcement above concluded that a “review of all available data at this time shows that the J&J/Janssen COVID-19 Vaccine’s known and potential benefits outweigh its known and potential risks”. Below is my interpretation of the benefits and risks of the J&J vaccine and how one can use the available data to weigh these factors against each other.
Please note, this analysis is based on several assumptions and is a simple, straightforward assessment of the data (the CDC/FDA certainly reviewed more factors when making their decision). I am not a doctor. This is not medical advice. The purpose of this blog post is to show how one can interpret the publicly available data to better understand the important developments around this major public health story.
- Recall that the absolute risk reduction in the number of COVID-19 cases in the J&J clinical trial was 1.7%.
- Assumption #1: For every 1,000,000 people that receive the J&J vaccine, let’s estimate (for the purposes of this exercise) that ~17,000 more people would have gotten COVID-19 had they not received the vaccine (using the 1.7% absolute risk reduction from the clinical trial to estimate).
- According to data from Johns Hopkins, the COVID-19 case-fatality rate (# of reported fatalities / # of documented cases) for the United States is 1.8% (Source: https://coronavirus.jhu.edu/data/mortality)
- Assumption #2: We can estimate the portion of individuals out of 1,000,000 that would have died of COVID-19 had they not received the vaccine (1,000,000 * 1.7% * 1.8% = 306).
- Based on the above estimates, we can make a case that the potential benefit of authorizing the use of the J&J vaccine would be in the order of magnitude of ~300 lives saved for every 1,000,000 people that receive the vaccine (this is meant to be a rough estimate used for comparison).
From the CDC again: “As of April 23, 2021, the reports reviewed all occurred in women between 18 and 59 years old, with a median of 37 years. These reports represent a reporting rate of 7 such events per 1 million vaccinations among women 18 through 49 years old and a rate of 0.9 per 1 million vaccinations among women 50 years and older. For all women, this is a rare adverse event. For women 50 years and older and men of all ages, the adverse event is even more rare.”
Based on the data, it is my understanding that the CDC concluded that the benefit of receiving the vaccine (as I estimate, it could save ~300 lives per 1,000,000 vaccine recipients) far outweighs the risk of blood clots (seen in ~7 out of every 1,000,000 women ages 18–49 who received the vaccine in the US).
A final note on the data — the estimate of number of lives above only considers those who actually received the vaccine. One of the major arguments of public health officials for people to get vaccinated is that it doesn’t only protect the vaccine recipient — it also helps stop the spread of the virus to others. Therefore, it is recommended that even healthy people who have a low risk of dying from COVID-19 should still get vaccinated. In addition, as more people get vaccinated, we, as a society, get closer to achieving herd immunity. In a future blog post, I’d like to explore what exactly is meant by “herd immunity” and how data science can be used to model what is necessary for achieving this important milestone in the fight against COVID-19.
