Errata: My original calculation underestimated the risk by a factor of about 2x. I neglected two key considerations, which fortunately somewhat canceled each other out. My new estimate from the calculation is 3.0 to 11.7 quality-adjusted days lost to long-term sequelae, with my all-things-considered mean at 45.
The two key things I missed:
- I estimated the risk of a non-hospitalized case is about 10x less than a hospitalized case, and so divided the estimates of disease burden by 10x. The first part is correct, but the second part would only make sense if all disease burden was due to hospitalized cases. In fact, there's a 15:85% split between hospitalized and non-hospitalized patients in the study (13,654:73,435). So if the disease burden for non-hospitalized is x, the total burden is 0.15*10x + 0.85*x = 2.35x. So we should divide by 2.35, not 10.
- However, as Owain pointed out below, the [demographics](https://www.nature.com/articles/s41586-021-03553-9/tables/1) are non-representative and probably skew high-risk given the median age is 60. the demographics are relatively high-risk. Indeed, this is suggested by the 15% hospitalized figure (which also, I suspect, means they just never included asymptomatic and most mildly symptomatic cases). An ONS survey (Figure 4) put symptoms reported after 5 weeks at 25% (20-30%) for 50-69 year olds and 17.5 (12.5 to 22.5%) for 17 to 24 year olds, which is surprisingly little difference, about a 1.5 decrease. I'd conjecture a 2x decrease in risk (noting that assuming no hospitalization is already doing a lot of work here).
Original post:
I did my own back-of-the-envelope calculation and came up with a similar but slightly higher estimated cost of 1.4 to 5.5 quality-adjusted days lost to long-term sequalea conditional on getting symptomatic COVID case. FWIW, I originally thought the OPs numbers seemed way too low, and was going to write a take-down post -- but unfortunately the data did not cooperate with this agenda. I certainly don't fully trust these numbers: it's based on a single study, and there were a bunch of places I didn't keep track of uncertainty, so the true credible interval should definitely be a lot wider. Given that and the right-tailed nature of the distribution, my all-things-considered mean is closer to 30 because of this, but figured I'd share the BOTEC anyway in case it's helpful to anyone.
My model is pretty simple:
1. What % of symptoms are there at some short-term follow up period (e.g. 4 to 12 weeks)? This we actually have data on.
2. How bad are these symptoms? This is fairly subjective.
3. How much do we expect these symptoms to decay long-term? This is going off priors.
For 1. I used Al-Aly et al (2021) as a starting point, which was based on comparing medical records between a COVID-positive and non-COVID demographically matched control group in the US Department of Veterans Affairs database. Anna Ore felt this was one of the more rigorous ones, and I agree. Medical notes seem more reliable than self-report (though far from infallible), they seem to have actually done a Bonferroni correction, and they tested their methodology didn't pick up any false positives via both a negative-outcome and negative-exposure controls. Caveat: many other studies have scarier headline figures, and it's certainly possible relying on medical records skews this low (e.g. doctors might be reluctant to give a diagnosis, many patients won't go to the doctor for mild symptoms, etc).
They report outcomes that occurred between 30 and 180 days after COVID exposure, although infuriatingly don't seem to break it down any further by date. Figure 2 shows all statistically significant symptoms, in terms of the excess burden (i.e. increase above control) of the reported symptom per 1000 patients. There were 38 in total, ranging from 2.8% (respiratory signs and symptoms) to 0.15% (pleurisy). In total the excess burden was 26%.
I went through and rated each symptom with a very rough and subjective high / medium / low severity. 2% excess burden of high severity symptoms, 19% medium severity, 5% low severity. I then ballparked that high severity (e.g. heart disease, diabates, heart failure) wiped out 30% of your QALYs, medium severity (e.g. respiratory signs, anxiety disorders, asthma) as 5% and low (e.g. skin rash) as 1%. Caveat: there's a lot of uncertainty in these numbers. Although I suspect I've gone for higher costs than most people would, since I tend to think health has a pretty big impact on productivity.
Using my weightings, we get a 1.6% reduction in QALYs conditional on symptomatic COVID case. I think this is misleading for three reasons:
1. Figure 3 shows that excess burden is much higher for people who were hospitalized, and if anything the gap seems bigger for more severe symptoms (e.g. about 10x less heart failure in people positive but not hospitalized, whereas rates of skin rash were only 2x less). This is good news as vaccines seem significantly more effective at preventing hospitalizations, and if you are fortunate enough to be a young healthy person your chance of being hospitalized was pretty low to begin with. I'm applying a 10x reduction for this.
2. This excess burden is per diagnosis, not per patient. Sick people tend to receive multiple diagnoses. I'm not sure how to handle this. In some cases, badness-of-symptoms does seem roughly additive: if I had a headache, I'd probably pay a similar amount not to also develop a skin rash then if my head didn't hurt. But it seems odd to say that someone who drops dead from cardiac arrest was more fortunate than another patient with the same cause of death, who also had the misfortune of being diagnosed with heart failure a week earlier. So there's definitely some double-counting with the diagnosis, which I think justifies a 2-5x decrease.
3. This study was presumably predominantly the original COVID strain (based on a cohort between March 2020 and 30 November 2020). Delta seems, per the OP, about 2-3x worse: so let's increase it by that factor.
Overall we decrease 1.6% by a factor of 6.5 (10*2/3) to 25 (10*5/2), to get a short-term QALY reduction of 0.064% to 0.24%.
However, El-Aly et al include any symptom reported between 30 to 180 days. What we really care about is chance of lifelong symptoms if someone is experiencing a symptom after 6 months there seems like a considerable chance it'll be lifelong, but if only 30 days has elapsed the chance of recovery seems much higher. A meta-review by Thompson et al (2021) seems to show a drop of around 2x between symptoms in a 4-12 week period vs 12+ weeks (Table 2), although with some fairly wild variation between studies so I do not trust this that much. In an extremely dubious extrapolation from this, we could say that perhaps symptoms half again from 12 weeks to 6 months, again from 6 months to a year, and after that persist as a permanent injury. In this case, we'd divide the "symptom after 30 days figure" from Al-Aly et al by a factor of 8 to get the permanent injury figure, which seems plausible to me (but again, you could totally argue for a much lower number).
With this final fudge, we get a lifelong QALY reduction of 0.008% to 0.03%. Assuming a 50-year life expectancy, this amounts to 1.4 to 5.5 days of cost from long-term sequelae. Of course, there are also short-term costs (and risk of morbidity!) that is omitted from this analysis, so the total costs will be higher than this.
I did actually mean 45, in "all-things-considered" I was including uncertainty in whether my toy model was accurate. Since it's a right-tailed distribution, my model can underestimate the true amount a lot more than it can overestimate it.
For what it's worth, my all-things-considered view for Delta is now more like 30, as I've not really seen anything all that compelling for long COVID being much worse than in the model. I'm not sure about Omicron; it seems to be less virulent, but also more vaccine escape. Somewhere in the 15-90 day range sounds right to me, I've not thought enough to pin it down precisely.
Curation notice by Raemon: Curated. It seems really important to figure out how to deal with the Delta variant, and how to interface with covid variants longterm. I appreciated this post for:
(Not a doctor; merely extrapolating lines of reasoning where they will go; all policy “recommendations” are entirely hypothetical and not actual recommendations; I rushed to finish this while it's still useful and relevant, so errors will exist and I apologize for the style)
Summary
As many of you know, the Delta strain of COVID is basically upon us. In the Bay, as of a few days ago, about .4% of people had it (based on .1% confirmed cases).
Currently it's doubling every week or so, but it's surprisingly difficult to tell if it will be a small blip or a huge wave. The main reason it's hard to tell is that it's entirely dependent on human behavior, which could change quickly if people become scared or if the government institutes rules again. The second reason it's hard to tell is because of mixed evidence on the difficulty of reining it in—while different studies makes this look more or less difficult, India and the UK both appear to have successfully done this without obviously draconian behavioral interventions. Berkeley might easily peak at 1% and drop back down, but for this post I'll be talking about policies for if we get a large wave.
We also don't know very much about vaccine efficacy against Delta. We know that the good vaccines reduce your risk by about 60-90% compared to without the vaccine, but the error bars are extremely large on all these studies. In part this is because it's so hard to disentangle effects from different strains, behavioral change in the vaccinated, social bubble effects, etc. Further, it doesn't answer the important question of whether this means ~80% of people are totally safe, or if it means that standing a few feet closer to people than you do now will cause the 5x difference that moots your protection. (This also plays into the above issue of predicting whether we will have a large increase or not.)
That being said, we can still roughly estimate risk from definitely having Delta. A healthy 30yo probably has about 4x (3x-10x) less risk than before, due to vaccination, despite Delta causing higher mortality. It almost entirely comes from Long COVID. In absolute terms this is ~4 expected days of life, plus 1/200th of your future productivity and vitality. You can shorthand this to about 1-4 weeks of life lost if you expect to otherwise live a full life—obviously, it costs less if you expect to live less less time. This translates to microCOVIDs at roughly 1 hour of your life lost every 1k-5k uCOVIDs. Risk of death goes up by 3x for every decade in age, but Long COVID probably only scales at ~~1.5x per decade, so for people over 60 mortality starts becoming more relevant. All these calculations are "creative" so please don't take them as definites.
By default it will probably take 3-12 months for this to resolve. I strongly believe we should not lock down again for this long—I think we need a return to some level of normalcy, plus the risks are much lower than before. I don't strongly advocate for a single policy in this document, since I don't know what other people's risk-tolerances are, but I do give a number of principles.
Personally, I am going to prioritize protecting the vulnerable—people with immune disruptions or age. I am probably going to advocate for them to get third vaccine shots of a different brand, if possible and if the downsides aren't too down. I also am going to start running the numbers on what a variolation scheme might look like this time around. I also am hoping that tight-knit communities of responsible people can beat out trivial inconveniences and put the requisite effort into creating safe spaces for socializing with tolerable risk levels—this is very prosocial, both figuratively and literally.
Delta vs originals:
Main sources I used:
Supporting sources:
Parameter estimation:
Cost of getting Delta:
Policy Principles:
Principle: Lockdowns last time were too strong
Principle: We don’t know the endgame
Principle: Very hard to predict the control system
Principle: Tests still suck
Principle: No one knows how to interpret vaccine “effectiveness” into the quantity we care about
Both of these worlds would show 90% "effectiveness, but for two different reasons. In one, all the variance comes from immune responsiveness. In the other, all the variance comes from environmental stochasticity, mostly from viral load. Currently, effectiveness numbers themselves give us no evidence for where on the axis between these worlds we are. Yet each have VERY different implications: if it’s entirely dependent on innate responsiveness, healthy people basically shouldn’t take protective measures since everyone’s going to get a small dose eventually, and it’s unlikely that you’re in the “definitely will get it” category. If effectiveness varies entirely with environment, you can never be safe, and acting normally is bound to eventually get you in contact with a 10^7 bolus that gets you very sick. (So we’d really prefer the world where variance is explained by immune responsiveness.)
We do have some evidence about where we are on this axis between the worlds. There are studies which find pretty big differences in level of antibody titer produced by the vaccinated, and in some cases where they have almost no antibodies it’s pretty clear that this means immune responsiveness is going to be at fault when they get sick. And I think there are studies finding correlation between titer and effectiveness. Both of these point toward innateness. But we also know that it has to be true that for many of those with low levels of antibodies, a larger dose will push them over the edge. There is also slight evidence from the Israel numbers, which give effectivenesses that vary some over time, that there’s a serious behavioral/environmental component. I don’t know of any good numbers personally, and it will be a question of distributions anyways that will end up hard to interpret. Could use more research
Principle: Variolation maybe good, still hard
Principle: Lockdowns are pretty mentally costly
Principle: Don't pay lots of COVID-avoidance costs if you're going to almost-surely get it anyways
Principle: A third vaccine dose of a different type is probably fairly useful for the vulnerable
Policy summary:
(These are for individuals, not the state. I would like the state to not impose any hard rules and instead just get us more and better tests, but I know that’s a pipe dream.)
As individuals, the primary behavioral axis is still how much risky contact you have with others, and using microCOVID.org to titrate your risk level at the new rate of 1k-5k uCOVIDs per hour. However, you have some secondary options this time around. First, you can get a third dose of a different vaccine (in Israel they’re already doing this); second, if you’re smart about it you can variolate yourself at less cost than before due to vaccine protection. More likely, a group of people working together could make variolation work.
On the primary behavioral axis, I think serious lockdowns are a bad idea for almost all people except the immunocompromised. I also think it wouldn't be crazy to just continue to act normal aside from reducing viral load in simple ways, and just avoid interacting with the vulnerable for a month or two until you’ve had Delta and been protected. However, I could be wrong about this and you should make your own decisions: I admit that I balk at taking a 1/200 permanent hit to productivity/vitality.
I had meant to make some more serious policy proposals for Bay Area rationalists or other groups of well-coordinated people, but I have to postpone this for a few days.
Again, remember that Delta might peak in the US in a few weeks anyways.
Research supported by LessWrong.