> Of course, the outcomes we’re interested in are hospitalization, severe Covid, and death. I’d expect the false positives on these to be lower than for having Covid at all, but across tens of thousands of people (the Israel study did still have thousands even in later periods), it’s not crazy that some people would be very ill with pneumonia and also get a false positive on Covid.
Does this observation undermine the claim of a general trend in effectiveness with increasing severity of disease? That is, if false positives bias the measured effectiveness downward, and if false positives are more frequent with less severe severe, then the upward trend is less robust and our use of it to extrapolate into places where the error bars are naively large is less convincing.
I thought about this for a while, and I think the entailment you point out is correct and we can't be sure the numbers turn out as in my example.
But also, I think I got myself confused when writing the originally cited passage. I was thinking about how there will be a smaller absolute number of false-positive deaths than the absolute number of false-positive symptomatic cases, because there are fewer death generally. That doesn't require the false-positive rates to be different to be true.
Also thinking about it, the mechanisms by which the false-posi... (read more)
I owe tremendous acknowledgments to Kelsey Piper, Oliver Habryka, Greg Lewis, and Ben Shaya. This post is built on their arguments and feedback (though I may have misunderstood them).
Update, May 13
I first wrote this post before investigating the impact of covid variants on vaccine effectiveness, listing the topic as a major caveat to my conclusions. I have now spent enough time (not that much, honestly) looking into variants that I have a tentative position I'm acting on for now.
My moderately confident conclusion is that the current spread of variants in the US barely impacts vaccine effectiveness. The Pfizer vaccine is reported to be 85% as effective against the feared B.1.351 variant (South African) as it is against B.1.1.7 (UK). Assuming that other variants are no more resistant than B.1.351 on average (a reasonable assumption) and that presently variants are no more than 25% of Covid cases (in Alameda and San Francisco). The net effect is 0.25*0.85 + 0.75*1.0 = 0.9625. In other words, vaccines still have 96% of the effect they would if B.1.1.7 were the only variant.
Plus, that tiny reduction of vaccine effectiveness is dwarfed by the falling background prevalence of Covid. When I first wrote this post, Alameda and San Francisco were at 0.1-0.15%; now they're at ~0.05%. The same for New York and the United Kingdom.
Although relaxing of restrictions might reverse this, right now, Covid-risk is very, very low in the Bay Area and many parts of the US.
All updates/changelog can be viewed here.
In its most extreme form, I have heard it claimed that the vaccines provide 10x reduction against regular Covid, 100x against severe Covid, and 1000x against death. That is, for each rough increase in severity, you get 10x more protection.
This makes sense if we think of Covid as some kind of "state transition" model where there's a certain chance of moving from lesser to more severe states, and vaccines reduce the likelihood at each stage.
I think 10x at multiple stages is too much. By the time you're at 1000x reduction, model uncertainty is probably dominating. I feel more comfortable positing up to 100x, maybe 500x reduction. I dunno.
There is a more limited claim of extreme vaccine effectiveness that I will defend today:
(Maybe it's 2x more effective against severe-Covid and 3x more effective against death compared to just getting it at all. Something like that, it doesn't have to be 10x– it'd still be a big deal because more severe outcomes are where most of the disutility lies.)
It's a very simple argument, really. First, the data very clearly suggests effectiveness of ~99% for young people, with nice tight confidence intervals. Second, across all the data we see trends of increasing effectiveness against increasing severity, granted that the confidence intervals are wide in some cases. Third, a very reasonable (imo) mechanistic model supports this interpretation of the data.
The 1.2 Million-Person Pfizer Israeli Observational Study
This observational study matched ~600k vaccinated people 1:1 with ~600k demographically similar controls. It covered the period December 20 to February 1. As far as I know, it is by far the largest Covid-19 vaccine study published to date. The other studies are clinical trials with sample sizes on the order of 20k-40k, and some other observational studies, typically with healthcare workers, in the single-digit thousands.
Why it's not as big as it sounds
Before we get to looking at data, I think it's important to note why uncertainty remains despite the huge N. To start with, the outcomes are all quite rare. Eyeballing it, Israel had a Covid-19 prevalence of ~0.5% during the study period. Out of a million people, a few thousand might be expected to actually catch Covid. Of that few thousand, only dozens or hundreds will progress to more severe forms of Covid. When sample sizes are in the dozens, confidence intervals are wide.
The authors report:
See Appendix B for complete breakdown of outcomes by period and vaccination status.
What's more, those numbers are for the entire study period (44 days). Only in a subset of days had participants been vaccinated long enough for it to be a real test. Nominally it is a 1.2 million person study, but practically, when you're looking at results 2 weeks after the first dose (~day 14) or one week after the second dose (~day 28), the numbers are much lower. ~80% lower.
All that to say, sample sizes aren't as big as they sound. Well, let's look at the results. This is the main outcome table. Definitions in Appendix A.
It's a bit hard to track trends formatted like that, so here's an equivalent graph:
Middle: 21-27 days after 1st dose;
Right: 7 days after 2nd dose until end of follow-up
To me, the headline result is that 2nd-dose + 7-days, vaccine effectiveness against Symptomatic Illness is 94% (87-98). Pretty good! Also, efficacy clearly rises from the earlier to later periods after vaccine administration.
Unfortunately, we don't see efficacy improvements moving to the right on the rightmost graph (more severe outcomes), counter to claims of extreme vaccine effectiveness. We do see upwards-right trends in the earlier periods (left and middle graphs).
Well, 2 out of 3 ain't bad! Too bad the last one is the one we care about most.
It's okay. I've got more. The astute reader will have noted that the above graphs have "subgroup = Full" in their title. The study made available endpoints (outcomes) for multiple subgroups. Below are some of them. (Here are ALL OF THEM.)
Since there was data for it, I also added in the "2nd dose and 6 days after" period.
Many of the points are missing. The authors did not compute vaccine effectiveness if the control and vaccine group combined did not have 10 or more instances of an outcome. For example, the Age 16-39 subgroup did not have 10 instances of hospitalization, severe-Covid, or death for almost all of the time periods.
In cases where there are 10 instances of an outcome in the control group but 0 in the vaccine group, a value is reported without confidence intervals, e.g. the dots in the Females Subgroup, 2nd-2nd+6 period.
The up-and-to-right shape of the graphs persists across subgroups, except for Males and when the values are already maxing out near 100% (the right-most graphs). Overall, I think this is suggestive of the general trend that vaccines are more effective against progressively more severe outcomes. I also suspect that an uncertainty modeling that took account of the not uncorrelated neighboring values would shrink the error bars beyond the naive bootstrap method.
I'll comment more on why the flat/missing trend in the rightmost graph doesn't bother me much, beyond the fact that sample size means it's hard for it to show much at all.
Also! If you prefer Tables, here's the top half from the paper itself corresponding to the above graphs:
As someone falling in the Age 16-39 subpopulation, I'm quite pleased to see 99% effectiveness against Symptomatic Infection, with a nice tight 96-100 confidence interval. This is higher than I'd had anyone cite to me, and is approximately a 5x increase in how effective I believe my Pfizer vaccine to be. That's even before we get to more severe outcomes.
So why is the absence of data showing increasing efficacy not evidence of absence?
Because we expect to see data that looks like this even in worlds where the vaccine is 100% effective (at least for all vaguely healthy people). To be evidence against something, it has to be less common in worlds where that thing is true, and that's not the case here.
Why would we see these numbers with a 100% effective vaccine?
"Saturation" and "noise"
When you have 100 true positives and 3 false positives, the false positives aren't such a big deal. When you have 0 true positives and 3 false positives, the false positives can change the entire picture.
I argue this is very likely what is going on with Covid-vaccine effectiveness, above and beyond sample sizes.
Consider that PCR Covid-19 tests have both a false negative and false positive rate (FPR). According to this random site I found by Googling that looks legit enough, the FPR for Covid-19 tests is between 0.2% and 0.9%. Let’s choose a point estimate of 1% for the FPR to be safe, but run it twice for every case to compensate. So now our FPR is 0.01%
Let’s now imagine using this test on a 99% effective vaccine (the same argument holds for 99.9% and 99.99% even more so). We run an RCT with 100,000 people receiving the vaccine and 100,000 receiving placebo. Covid-19 prevalence is 0.1% in the region our hypothetical test is running.
99 people from the control group catch actual Covid and receive positive test results (we lose one to a realistic false-negative rate of 10%, run twice to become 1%) plus 0.01% false positives for a total of 109. From the treatment group with 99% effective vaccine, we get 1 true-positive and 10 false-positive test results. Our final effectiveness estimate is 1 - 11/109 = 90%
90%! And that’s from what is in truth a 99% effective vaccine. The control is mostly unaffected by the noise (109 vs actual 100) but the treatment is enormously changed. Instead of 1, it’s 11.
The greatest noise of all is selection effects
Never mind fluke false positive tests, on priors we have reason to suspect that the people who are still getting severe Covid despite being vaccinated or even hospitalized are very likely not like you. Why isn’t the number of vaccinated people who ended up in critical condition zero?
Because within the Israel Observation study, the vaccinated group contains some very sick people. (In a half-million person group without exclusion criteria against it there simply will be, and that’s before the fact that we know many people are elderly and many explicitly meet risk criteria such as cancer, obesity, pulmonary disease, Type 2 diabetes, etc. See Table 1. Demographic and Clinical Characteristics of Vaccinated Persons and Unvaccinated Controls at Baseline.)
Vaccines work by stimulating an immune response. If your immune system is in tatters, unable to manufacture healthy antibodies or something, your vaccine might not do much. You might fare little better than the unvaccinated.
There’s actually a term for immune system failure in the elderly: immunosenescence. A couple of papers on that topic: Immunosenescence and vaccine failure in the elderly (2009) and Immunosenescence: A systems-level overview of immune cell biology and strategies for improving vaccine responses (2019)
Quoting the abstract from the first one:
The last line there suggested antibodies can be present even when a vaccine overall is not protective against disease.
Don’t get old kids, it’s bad for you.
When I see that the treatment group has lower but not zero severe cases than the control group, I assume it’s coming from the very ill, the immunocompromised and immunosenescent.
I am pretty confident that I am not in those groups. I’m pretty sure my vaccine elicited some real response (I had some side effects, not terrible, but some). When I see the effectiveness numbers showing globally that there’s still some chance of really bad outcomes, I adjust them downwards because they were very likely not happening to people with remotely my level of health.
To invoke the math of the previous section regarding false positives, the immunocompromised might be only 10% to the number of severe Covid patients in the control group (because their prevalence is low), but after the filter/selection effect of vaccination, they make up 95% of severe cases in treatment group.
Here’s a diagram for good measure:
This is a causal diagram. Arrows indicate the direction of causality but not the specific causal relationship. (Having a poor immune system, e.g. because you’re older, might make you more likely to be vaccinated, but of course, in our study we’re conditioning on vaccination status, so it doesn’t matter.)
Since we didn’t condition on immune system health, we can’t expect that a naive interpretation of the table of efficacies (above) tells us the true relationship between vaccines and outcomes.
Well, we approximately can condition. The vaccine efficacy is higher for the Age 16-39yr subgroup (99%, 96-100) vs the entire population (94%, 87-98). It's not just that younger people get less Covid, but that the vaccine worked better on them.
The Age 16-39yr subgroup didn't have zero cases of symptomatic disease (it might have zero of hospitalization, etc), but as above, I'm guessing those were almost all people who knowable had weak immune systems within the overall healthier group too.
Priors & Trends
In the above section I argued two things:
In this section, I want to offer 1) a plausible mechanistic model for why this should be true, 2) further indications of increasing effectiveness from other trials and studies.
Shifting the Distribution of “Infection”/Viral Load
Epistemic status: I'm not a biology/virology/immunology person and this feels kinda hand-wavy to me.
This post started with a table that lists a progression of discrete outcomes: Documented Infection, Symptomatic, Hospitalization, Severe Disease, Death . . . and I’ve been referring to them since. Obviously, the underlying biological reality isn’t quite as discrete as that.
It’s probably more something like there’s a continuous value of how infected you are, and the higher that value gets, the worse your condition will be.
How infected you are is probably a fuzzy thing in reality, but viral load might be an adequate proxy. It’s been documented that viral load varies together with Covid-19 severity. See SARS-CoV-2 viral load is associated with increased disease severity and mortality and Saliva viral load is a dynamic unifying correlate of COVID-19 severity and mortality
The graphs in the second paper particularly make this point.
Presumably, your viral load is the result of competition between the virus replicating and your immune system fighting it. Vaccination gives a significant boost to your immune system. (Cf the oft-cited claim about “4x lower” viral load in vaccinated people)
!! To engage in some inexpert armchair speculation, I’d guess that in the virus-immune system race, about x% of the time the virus gets the upper hand with an infected person and makes them symptomatic (~Level 1), and then in x% of cases where it got to Level 1, the virus wins out again to progress to Level 2 before the immune system can stop it. The virus progresses two levels x^2% of the time. If x is 20%, then overall 4% to get to two levels worse.
At each level, the virus only has an x% of winning out and progressing to the next level. Alternatively, in each time period, there’s some chance, y%, that the immune system will catch up and win.
In cases where someone has a weak immune system (high x%, small y%), increasing levels of case severity aren’t much less likely than earlier ones. You might get an approx flat effectiveness curve.
But suppose that someone is vaccinated and has a real boost to their immune system. Intuitively, I’d reckon they’re now at x/5% for each stage. For the virus to progress two levels, it’s (x/5)^2, or 0.16% when x=20%.
Maybe it’s a factor 2 or 3 instead of 5, but either way, it’d be a compounding effect. More severity means the virus has to replicate more times, which is more time for the immune system to catch up and beat it, ergo less chance for it to get that bad.
It's tough being a dude
Incidentally, Silva et al (2021) who provided the graphs of viral load immediately above, also had this to say re male vs female:
The difference in viral loads lines up with the Male subgroup having worse vaccine efficacy than the Female subgroup: 88% (71-98) vs 96% (90-100) against symptomatic infections, 2nd dose+7 . It is also the case that across the world, women live longer and die less often from cardiovascular disease, cancer, diabetes, and chronic respiratory disease (Our World in Data).
Johnson & Johnson & Friends
Although not an mRNA vaccine like today's star, Pfizer's BNT162b2, the J&J clinical trial tracked multiple endpoints that show our hoped-for trend.
The design/reporting of the J&J clinical trial differs from others, particularly the large Pfizer observational study. The time periods and outcomes are defined differently.
To avoid copying multiple tables, I’ve extracted the numbers as I’ve understood them.
Onset at Least 14 Days
Onset at Least 28 Days
18-59 yrs
>=60 years
18-59 yrs
/Critical,
>= 60 years
There were deaths in the J&J study, all within the placebo group:
Using a somewhat different formula, the authors also report on interim asymptomatic results. They present four different operationalizations of which I choose two, the ones with the highest and lowest efficacy after 29 days. See Table 20 for further detail.
Day 1-Day 29
After Day 29
Despite the J&J trial using overlapping criteria, both explicitly lumping things and in their criteria, we see the progression we’d expect to see if vaccines work better to prevent worse outcomes than they do milder ones. Modulo confidence intervals, that is.
While efficacy against moderate to severe Covid-19 is 65% (for 18-59 years old), it jumps to 85% for severe alone, granted the overlap in confidence intervals. 0 cases in the vaccine group fell into the Requiring Medical Intervention endpoint, compared to 7 with placebo. It’s as good as we could hope to see with this data.
However, there isn’t a clear jump between “asymptomatic” and “moderate to severe”. Partly because the operationalization isn’t clear (59.7% to 66% is a jump) but there are still wide error bars. The After Day 29 antibody test was conducted at Day 71 and had only been completed for ~30% of participants at the time of publication.
On net, I think the overall endpoint trend lines up with increasing vaccine efficacy against more severe outcomes, but the asymptomatic vs symptomatic doesn't clearly show it, but part of that is I don't really understand the groups or how to interpret them.
Let's get clinical, clinical
Moderna Clinical Trial
The Moderna Phase III Trial tracked Covid and severe Covid but no other end points. There were no severe Covid or deaths in the vaccine group, but 30 severe Covid cases in control and one death. Doesn’t let you compute a precise value, but is consistent with the vaccines being very good.
Pfizer Clinical Trial
It’s nice (for me) to note that the vaccine efficacy, and particularly the confidence intervals, are higher for the 16-55yr group. 95.6% with CI 89.4%-98.6%. Doesn’t prove the main point, but is in line with the immunosenescence model.
No Covid deaths are reported in either placebo or vaccine groups. Table S5 from the appendix least severe-Covid outcomes. A total of 9 after the first dose in control, 1 in the vaccine group for a 89% reduction with confidence interval between 20.1%-99.7%. If we break it down to different time periods (before/after 1st/2nd dose), we end up with confidence intervals (-3800% to 100%). Yes, maybe taking the vaccine will increase your chance of severe-Covid by 39x!
As expected, the data doesn’t show that the reduction in severe-Covid is greater than lesser-Covid, but it also they doesn’t show the opposite either.
As described above, we have to note the greatly reduced sample size in the later periods.
Evidence of Increased Asymptomatic/Symptomatic Ratios
This was already shown in the mass Pfizer study, but several other sources indicate the ratio of asymptomatic-to-symptomatic cases is increased for vaccinated people. In other words, vaccination works better against symptomatic Covid (more severe) than asymptomatic Covid (less severe). This is at earlier side of "severity", compared to hospitalization, severe-Covid, and death, but it suggests the same trend–plus there's more data than when looking at more severe outcomes.
I've copied most of the numbers here form MicroCOVID.org, see their analysis for calculation detail.
Unvaccinated
Vaccinated
Symptomatic
vs
Asymptomatic
%
Asymptomatic
Symptomatic
vs
Asymptomatic
%
Asymptomatic
210 vs 191
48%
31 vs 59
65%
MicroCOVID Moderna/CDC
Calculation
185 vs 37
17%
11 vs 11
50%
351 vs 182
34%
117 vs 159
57%
248 vs 73
23%
84 vs 57
40%
Pfizer Asymptomatic percentage goes from 48%->65%, Moderna from 17%->50%, J&J from 34%-57%, AstraZeneca 23%->40%
I expect the very different absolute numbers to come from the widely varying study methodologies as much as differences between the vaccines. (AstraZeneca uses home tests, for example.)
I didn't exhaustively look through all possible sources of asymptomatic vs symptomatic efficacy. I would be very interested if someone had a credible source not showing this trend.
I also didn't scrutinize these calculations much, so I wouldn't be that surprised if it turned out there were deep flaws that undermine the trend seen here.
It gets better
If we go back to the big Israeli Pfizer observational study, we see increasing vaccine effectiveness as more time passes since first/second dose. Unfortunately, the study didn't have enough time/data to show us things two weeks after the 2nd dose.
Fortunately, there was a follow-up. On March 11, Pfizer/Israel Ministry of Health made the following press release:
The lack of actual paper makes this a little harder to interpret, but I don’t find it surprising given that (1) at this later date in their roll-out, an even greater proportion of people will be young and healthy, (2) this data is only counting two weeks after the second dose, whereas the previous large observational study only had a “7 days after 2nd dose until end of follow-up” (maximum of day 28 to 44).
And, of no small significance, the Pfizer vaccine appears fully effective against the UK variant. (Yay!!)
If the vaccine is showing 97+% amongst everyone, I would expect that's at least as true when you filter for younger/healthier people and filter out those with comorbidities.
What I believe
I believe that what I wrote above supports my initial assertions:
The initial Pfizer mass study has 99% (96-100) for the age 16-39yr group, and the subsequent follow-up gives 97% for everyone. At baseline for symptomatic cases, we're talking 30-100x reductions, which is hella good.
Further, across multiple studies, vaccines, and outcomes we see trends of increasing effectiveness against progressively severe outcomes. In some cases, we don't definitively see it, but that's easily attributable to lack of sample size and inherent limitations in methodology due to noise and selection effects.
If we're talking 99% against symptomatic cases (100x reduction), then I think it's reasonable to expect at least that for hospitalization, 99.5% (200x). Hence the title, extreme vaccine effectiveness.
What about J&J (and AstraZeneca)? Granted, The effectiveness numbers for J&J look lower than for Pfizer and Moderna, but I think they're higher than MicroCOVID.org's numbers imply. First, we get 85% (61.2-95.4) effectiveness against Severe/Critical in the 18-59yr subgroup. That number matters more. Second, that is a very wide age range. I would bet that restricting it to 18-39 would show an improvement relevant to most of those reading this. Lastly, I suspect all the factors mentioned above (selection effects, noise/saturation) to affect it and make the result lower than it would be otherwise.
On net, J&J might not be quite as extremely effective as the mRNA vaccines, but it's no pushover either.
AstraZeneca isn't on offer in the Bay, and was recently abandoned in my home country of Australia too, so I apologize for not examining it.
Tell me where I’m wrong
I want the case I've made today to be true, but even more than that I want to believe true things (and I certainly don't want people to believe false things because of me). If you think any of this is wrong. PLEASE SAY SO.
Many thanks!
Objections
Maybe the vaccine and control groups are different?
The Mass Israeli observation study is not an RCT. Instead, it's looking at their database of vaccinated people and then trying to match each of them to someone who's mostly the same except not vaccinated.
This might be hard to do. The kinds of people who seek out vaccines ("health-seeking" behavior) are probably a bit different from those who haven't gotten around to it. Maybe they wear better masks (or masks at all) on the hand, and maybe they risk-compensate on the other.
The authors attempt to control for this by 1:1 matching vaccinated people and controls on multiple characteristics. They cite as evidence this worked the fact the Covid cases among vaccinated and non-vaccinated groups are the same before Day 12 (before the vaccine really starts working), and explain the initial gap in the first couple of days as a selection effect, because vaccinated people only choose to get vaccinated when they're feeling well.
I didn't look at the data especially long, but I'm not sure what to make of it. If vaccinated people are more cautious, even after being vaccinated, this means vaccines are less effective than we'd otherwise think. If they risk-compensate, that means vaccines are more effective than what we observe.
I'm not sure how this should net out, but I do think it should widen confidence intervals somewhat.
I also expect the effect to get stronger over time in Israel, as the people who remain unvaccinated and serve as controls (in observational studies) who least cared about Covid and getting vaccinated. So...maybe that explains the later 97% effectiveness result.
Ben Shaya's Thoughts
Ben Shaya, one of the people responsible for MicroCOVID.org's models, has been kindly taking time to discuss the topic of vaccine effectiveness with me.
Here wrote a document arguing Why I think vaccines don’t bring the chance of severe COVID to 0. While that's a stronger claim than I would make, his arguments and models are still valuable when thinking about the topic generally.
Read them all in his doc, but I'll highlight one that I found very relevant to mechanistic models of Covid severity:
This gets at more gears in immune response and is the kind of thing that can expose where simple state transition models and immune response as a single thing don't hold up.
What if the vaccine boost one part of the immune system but not another? In that case, you might see the vaccine be very effective against symptomatic and asymptomatic Covid, but not more severe disease. If the virus makes its way deep into your lungs, and the lungs are protected by a different immune response that isn't helped by the vaccine– then, conditional on having gotten to that point, you might not be better off than a non-vaccinated person.
All this to say we should be cautious in putting too much stock in simple mechanistic models.
What if vaccines are all or nothing?
The model behind the claim of extreme vaccine efficacy is that even if your post-vaccine immune response isn’t enough to stop you getting Covid at all, it should be stronger than it would have otherwise been, and you’ll do better at fighting off severe-Covid. This takes vaccine efficacy as a continuous thing.
But maybe the vaccine is 100% effective against all outcomes! So long as it’s correctly transported and administered, that is. Except sometimes vaccines are left at high temperature for too long, the delicate proteins are damaged, and people receiving them are effectively not vaccinated. If this happens 5% of the time, then 95% of people are completely immune to Covid and 5% are identical to not be vaccinated. Whatever chance they had of getting severe Covid before, it’s the same now.
In this world, not knowing whether your vaccine was a dud or not, post-vaccine you should assume you have a 95% reduction across all outcomes equally.
I originally found this argument very persuasive. How could assume that the “continuous” model of vaccine-immune response was true? But actually, most things are continuous in the real world, particularly in biology. People aren’t old or young, but somewhere on a continuous measure. Immune response isn’t an all or nothing affair, and some people’s body’s will produce more antibodies than others. Some enough to stymy any symptoms at all, but some only enough to prevent them getting hospitalized.
One person told me that the mRNA vaccines induce 60x the antibodies of a recovered Covid patient (closest source I found), such that even if your response was weaker, it should still be more than powerful to deal with any actual Covid. Therefore, we should take people still getting quite sick with Covid as a sign that some people’s vaccines must not be working at all.
I would be surprised if complete vaccine failure didn't occur some of the time, the question is how much. Suppose we have a vaccine that, when it works successfully, confers 100x reduction (99% efficacy). If it fails to work at all 5% of the time, we'd see a
1-(0.01*0.95 + 1*0.05) = 94% efficacy overall. Something like that could be a big part of what we observe.
Also, Kelsey Piper says that she couldn't find anything about any other vaccines working this way (all or nothing). [Edited: However, as Romeo points out in the comments, Moderna and Pfizer are mRNA vaccines that might not behave like the older types.]
Caveats – read before you act
In my long case for the extreme effectiveness of vaccines, there are some topics of crucial practical importance. These imply that maybe you don't want to throw caution to the wind just yet. I'm not sure, I didn't get to looking into these.
Long-Covid
Long-Covid is not an endpoint tracked by any of the studies I've looked at. I would think that'd be related to increased viral load and behave like an outcome more severe than just a symptomatic case, but there isn't data for that. Anecdotally, I've heard of a couple of cases where someone experienced mild Covid yet was dramatically affected for months afterwards.
My conservative gut estimate is that your odds or getting long-Covid are reduced by a vaccine by as much as your chance of getting symptomatic Covid at all, but not necessarily any more than that.
Variants
The Israeli study showed supreme efficacy even against the much-feared UK variant (B.1.1.7). However, there are fears the vaccines aren't not nearly as good against the South African variant (B.1.351) or Brazilian P.1 variant.
A new study published on MedRxiv last week, Evidence for increased breakthrough rates of SARS-CoV-2 variants of concern in BNT162b2 mRNA vaccinated individuals, state that vaccinated individuals were more likely to contract B.1.1.7 and B.1.351 than unvaccinated controls, suggesting both variants are more resistant to vaccination than other strains. (Caveat: I didn't read the paper in detail.)
Not good.
The CDC's CovidTracker page actually has some nice dashboards for tracking variant proportion, though I haven't looked into the data quality. Their brief is helpful too.
There's a breakdown by US state too. I'm in California and am pleased to see that currently, B.1.351 is only 0.3% of the Covid cases and P.1 is 1.6%
Of course, if B.1.351 and friends are resistant to vaccination, we will see them rise in prevalence.
This needs more investigation. Without looking into it more, the sensible strategy would be something like act according to your local prevalences and beliefs about how vaccine-resistant the different strains are. Right now the suspicions are on B.1.351 and P.1, but they're uncommon in California (0.3% and 1.6% respectively).
If you've got the time and skill to look into this more, please do, I can provide you money and glory.
Spreading to the Unvaccinated
Even if the vaccine protects you 200x against more severe outcomes, that doesn't help the unvaccinated if they catch Covid from you when you had an asymptomatic or mild case. This means that until such time as those you interact with most are vaccinated, you might want to be more conservative in your microCovids.
MicroCOVID.org calculated reductions in contagiousness for vaccinated people (10x reduction for Pfizer/Moderna, 3x for J&J), and that's what I'd stick to myself if interacting with the unvaccinated. (But hey, California is just about at universal eligibility, now is the time!)