Delta Strain: Fact Dump and Some Policy Takeaways
39AdamGleave
6Ben Pace
3habryka
3Ben Pace
2habryka
3Connor_Flexman
11AdamGleave
5AdamGleave
22Connor_Flexman
7Owain_Evans
3Connor_Flexman
5Owain_Evans
3Connor_Flexman
3Owain_Evans
5AdamGleave
2DanielFilan
3AdamGleave
1MichaelStJules
17James Payor
15Connor_Flexman
5Connor_Flexman
6James Payor
7Owain_Evans
2j2b
1James Payor
6James Payor
7Zac Hatfield-Dodds
1TheSimplestExplanation
2Zac Hatfield-Dodds
14Raemon
13Owain_Evans
6GDorn
4Owain_Evans
1Connor_Flexman
13Richard_Kennaway
37JenniferRM
5Connor_Flexman
20JenniferRM
3TheSimplestExplanation
1Connor_Flexman
1TheSimplestExplanation
7Daniel V
4Connor_Flexman
6ESRogs
4gjm
2gjm
6ChristianKl
7Owain_Evans
6Ben Pace
4TurnTrout
3Connor_Flexman
4Roko
3Neel Nanda
4Connor_Flexman
3DanielFilan
1Connor_Flexman
2Sherrinford
2abramdemski
1Connor_Flexman
1Connor_Flexman
New Comment
Errata: My original calculation underestimated the risk by a factor of about 2x. I neglected two key considerations, which fortunately somewhat canceled each other out. My new estimate from the calculation is 3.0 to 11.7 quality-adjusted days lost to long-term sequelae, with my all-things-considered mean at 45.
The two key things I missed:
- I estimated the risk of a non-hospitalized case is about 10x less than a hospitalized case, and so divided the estimates of disease burden by 10x. The first part is correct, but the second part would only make sense if all disease burden was due to hospitalized cases. In fact, there's a 15:85% split between hospitalized and non-hospitalized patients in the study (13,654:73,435). So if the disease burden for non-hospitalized is x, the total burden is 0.15*10x + 0.85*x = 2.35x. So we should divide by 2.35, not 10.
- However, as Owain pointed out below, the [demographics](https://www.nature.com/articles/s41586-021-03553-9/tables/1) are non-representative and probably skew high-risk given the median age is 60. the demographics are relatively high-risk. Indeed, this is suggested by the 15% hospitalized figure (which also, I suspect, means they just never included asymptomatic and most mildly symptomatic cases). An ONS survey (Figure 4) put symptoms reported after 5 weeks at 25% (20-30%) for 50-69 year olds and 17.5 (12.5 to 22.5%) for 17 to 24 year olds, which is surprisingly little difference, about a 1.5 decrease. I'd conjecture a 2x decrease in risk (noting that assuming no hospitalization is already doing a lot of work here).
Original post:
I did my own back-of-the-envelope calculation and came up with a similar but slightly higher estimated cost of 1.4 to 5.5 quality-adjusted days lost to long-term sequalea conditional on getting symptomatic COVID case. FWIW, I originally thought the OPs numbers seemed way too low, and was going to write a take-down post -- but unfortunately the data did not cooperate with this agenda. I certainly don't fully trust these numbers: it's based on a single study, and there were a bunch of places I didn't keep track of uncertainty, so the true credible interval should definitely be a lot wider. Given that and the right-tailed nature of the distribution, my all-things-considered mean is closer to 30 because of this, but figured I'd share the BOTEC anyway in case it's helpful to anyone.
My model is pretty simple:
1. What % of symptoms are there at some short-term follow up period (e.g. 4 to 12 weeks)? This we actually have data on.
2. How bad are these symptoms? This is fairly subjective.
3. How much do we expect these symptoms to decay long-term? This is going off priors.
For 1. I used Al-Aly et al (2021) as a starting point, which was based on comparing medical records between a COVID-positive and non-COVID demographically matched control group in the US Department of Veterans Affairs database. Anna Ore felt this was one of the more rigorous ones, and I agree. Medical notes seem more reliable than self-report (though far from infallible), they seem to have actually done a Bonferroni correction, and they tested their methodology didn't pick up any false positives via both a negative-outcome and negative-exposure controls. Caveat: many other studies have scarier headline figures, and it's certainly possible relying on medical records skews this low (e.g. doctors might be reluctant to give a diagnosis, many patients won't go to the doctor for mild symptoms, etc).
They report outcomes that occurred between 30 and 180 days after COVID exposure, although infuriatingly don't seem to break it down any further by date. Figure 2 shows all statistically significant symptoms, in terms of the excess burden (i.e. increase above control) of the reported symptom per 1000 patients. There were 38 in total, ranging from 2.8% (respiratory signs and symptoms) to 0.15% (pleurisy). In total the excess burden was 26%.
I went through and rated each symptom with a very rough and subjective high / medium / low severity. 2% excess burden of high severity symptoms, 19% medium severity, 5% low severity. I then ballparked that high severity (e.g. heart disease, diabates, heart failure) wiped out 30% of your QALYs, medium severity (e.g. respiratory signs, anxiety disorders, asthma) as 5% and low (e.g. skin rash) as 1%. Caveat: there's a lot of uncertainty in these numbers. Although I suspect I've gone for higher costs than most people would, since I tend to think health has a pretty big impact on productivity.
Using my weightings, we get a 1.6% reduction in QALYs conditional on symptomatic COVID case. I think this is misleading for three reasons:
1. Figure 3 shows that excess burden is much higher for people who were hospitalized, and if anything the gap seems bigger for more severe symptoms (e.g. about 10x less heart failure in people positive but not hospitalized, whereas rates of skin rash were only 2x less). This is good news as vaccines seem significantly more effective at preventing hospitalizations, and if you are fortunate enough to be a young healthy person your chance of being hospitalized was pretty low to begin with. I'm applying a 10x reduction for this.
2. This excess burden is per diagnosis, not per patient. Sick people tend to receive multiple diagnoses. I'm not sure how to handle this. In some cases, badness-of-symptoms does seem roughly additive: if I had a headache, I'd probably pay a similar amount not to also develop a skin rash then if my head didn't hurt. But it seems odd to say that someone who drops dead from cardiac arrest was more fortunate than another patient with the same cause of death, who also had the misfortune of being diagnosed with heart failure a week earlier. So there's definitely some double-counting with the diagnosis, which I think justifies a 2-5x decrease.
3. This study was presumably predominantly the original COVID strain (based on a cohort between March 2020 and 30 November 2020). Delta seems, per the OP, about 2-3x worse: so let's increase it by that factor.
Overall we decrease 1.6% by a factor of 6.5 (10*2/3) to 25 (10*5/2), to get a short-term QALY reduction of 0.064% to 0.24%.
However, El-Aly et al include any symptom reported between 30 to 180 days. What we really care about is chance of lifelong symptoms if someone is experiencing a symptom after 6 months there seems like a considerable chance it'll be lifelong, but if only 30 days has elapsed the chance of recovery seems much higher. A meta-review by Thompson et al (2021) seems to show a drop of around 2x between symptoms in a 4-12 week period vs 12+ weeks (Table 2), although with some fairly wild variation between studies so I do not trust this that much. In an extremely dubious extrapolation from this, we could say that perhaps symptoms half again from 12 weeks to 6 months, again from 6 months to a year, and after that persist as a permanent injury. In this case, we'd divide the "symptom after 30 days figure" from Al-Aly et al by a factor of 8 to get the permanent injury figure, which seems plausible to me (but again, you could totally argue for a much lower number).
With this final fudge, we get a lifelong QALY reduction of 0.008% to 0.03%. Assuming a 50-year life expectancy, this amounts to 1.4 to 5.5 days of cost from long-term sequelae. Of course, there are also short-term costs (and risk of morbidity!) that is omitted from this analysis, so the total costs will be higher than this.
(I'd personally appreciate you saying how many microcovids you think is equivalent to an hour's time; that's the main number I've been using to figure out whether various costs are worth it.)
The calculation would be pretty straightforward:
3.5 days mean, let's assume another 2 days or so in terms of sickness, so 5.5 days lost in total for 1M microcovids.
5.5 days = 130 hours
Which implies that ~7600 (1MM / 130) microcovids is ~1 hour of life lost.
So Adam is saying microcovids are cheaper than the OP does? Connor writes
1 hour of your life lost every 1k-5k uCOVIDs
(Pardon me commenting while not getting into the details on this important topic, I am busy but am trying to track at least whether there's disagreement and in what direction.)
According to this specific calculation. He does say his all things considered view is like 5x the cost.
So, to be clear, his all-things-considered view is about 1.5k uCOVIDs cost an hour, which is toward the edge of my range that corresponds to high risk, while his numerical estimate is at 7.5k uCOVIDs costing an hour, which is just outside of my range that corresponds to low risk (but matches almost exactly with my estimates of Long COVID risk outside of follow-ons from cognitive impairment!). So it sounds like initially he disagreed toward higher risk, then found very similar numbers as I did, now only leans toward the high end of my risk estimate due to priors and right-tail uncertainty. (Apologies if my paraphrase does not do your view justice, Adam.)
This is an accurate summary, thanks! I'll add my calculation was only for long-term sequelae. Including ~10 days cost from acute effects, my all-things-considered view would be mean of ~40 days, corresponding to 1041 uCOVIDs per hour.
This is per actual hour of (quality-adjusted) life expectancy. But given we spend ~1/3rd of our time sleeping, you probably want to value a waking-hour at 1.5x a life-hour (assuming being asleep has neutral valence). If you work a 40 hour work week and only value your productive time (I do not endorse this, by the way), then you'd want to adjust upwards by a factor of (7*24)/40=4.2.
However, this is purely private cost. You probably want to take into account the cost of infecting other people. I'm not confident in how to reason about the exponential growth side of things. If you're in a country like the US where vaccination rates have plateaued, I tend to expect Delta to spread amongst unvaccinated people until herd immunity is reached. In this scenario you basically want infection rates to be as high as possible without overwhelming the healthcare system, so we get to herd immunity quicker. (This seems to actually be the strategy the UK government is pursuing -- although obviously they've not explicitly stated this.) But if you're in a country that's still actively vaccinating vulnerable people, or where flattening the curve makes sense to protect healthcare systems, then please avoid contributing to exponential growth.
Neglecting the exponential growth side of things and just considering immediate impact on your contacts, how likely are you to transmit? I'd be surprised if it was above 40% per household contact assuming you quarantine when symptomatic (that's on the higher end of transmission seen even with unvaccinated primary cases), but I'd also be surprised if it was below 5% (lowest figure I've seen); I'd guess it's around 15% for Delta. This means if you have ~6-7 contacts as close as housemates, then your immediate external cost roughly equals your private cost.
Specifically, two studies I've seen on secondary attack rate given vaccination (h/t @Linch) give pretty wildly varying figures, but suggest at least 2x reduction in transmission from vaccination. Layan et al (2021) found 40% of household contacts of Israeli medical staff developed an infection (when Alpha was dominant), with vaccination of the primary case reducing transmission by 80%, so an 8% chance of transmission overall. Harris et al (2021) from Public Health England suggest vaccination cuts transmission risk from 10% to 5%, but these figures are likely skewed low due to not systematically testing contacts.
Just to flag I messed up the original calculation and underestimated everything by a factor of 2x, I've added an errata.
I'd also recommend Matt Bell's recent analysis, who estimates 200 days of life lost. This is much higher than the analysis in my comment and the OP. I found the assumptions and sources somewhat pessimistic but ultimately plausible.
The main things driving the difference from my comment were:
- Uses data from the UK's Office of National Statistics that I'd missed, which has a very high number of 55% of people reporting symptoms after 5 weeks, with fairly slow rates of recovery all the way out to 120 days post-infection. Given this is significantly higher than most other studies I've seen, I think Matt is being pessimistic by only down-adjusting to 45%, but I should emphasize these numbers are credible and the ONS study is honestly better than most out there.
- Long COVID making your life 20% worse is on the pessimistic end. I put most mild symptoms at 5% worse. Ultimately subjective and highly dependent on what symptoms you get.
- I think the difference in hospitalized vs non-hospitalized risk is closer to 10x (based on Al-Aly figure) not Matt's estimate of 2x, that means we should multiply by a factor of ~60% not ~97%.
I should probably argue with Matt directly, but my brief take is that this is just entirely incompatible with what we see on the ground. The friends of mine who got COVID aren't reporting 45% chance of their life being 20% worse. That's... an incredibly massive effect that we would definitely see. Would anyone realistically bet on that?
Bell mentions this paper in Nature Medicine that finds only 2.3% of people having symptoms after 12 weeks. (The UK ONS study that is Bell's main sources estimates 13%). It seems better to take a mean of these estimates than to just drop one of them, as the studies are fairly similar in approach. (Both rely on self-report. The sample size for the Nature paper is >4000).
Note that the 13% figure in the ONS study drops to 1% if you restrict to subjects who had symptoms every week. (The study allows for people to go a week without any symptoms while still counting as a Long Covid case). I realize people report Long Covid as varying over time, but it's clearly worse to have a condition that causes some fatigue or tiredness at least once a week rather at least once every two weeks.
Great paper, thank you!
(Do you mean the Lancet / British intelligence test paper when you say ONS? I embarrassingly don't see a paper I cited with those letters in it.)
The current way I imagine citing this is to use as a corroboration of my rough estimate of <2% 30yos having Long COVID. I don't see an easy way to integrate it with IQ loss estimates—since I wouldn't expect tiny levels of IQ loss to show up on a survey about actual Long COVID symptoms, it seems relatively consistent for 2% symptoms after 12 weeks to still correspond to an average IQ loss of .15 points after 12 weeks (~10% lose 1 IQ point, 1% lose several IQ points). I do think it points downwards somewhat, though, maybe a factor of 2?
I added a link above. The ONS is the UK's national statistics agency. This is not a peer-reviewed paper but a report they published. (I find these reports to be mixed in quality).
In the Nature paper, they get 2.3% with symptoms overall. But they estimate that 30 yos are less likely than older cohorts to have symptoms at 56 days and so you could adjust down a bit. (Women are also at higher risk according to this study).
Oops, thought that was a top-level reply to me when I clicked on it, rather than a reply to Adam. Sorry. Makes more sense in context.
I quickly skimmed the El-Aly et al paper. It does look much better than some of the other studies. One concern is the demographics of the patients. Only 25% of people with Covid are younger than 48. Only 12% are female. I'd guess the veterans under 35 are significantly less affluent than LW readers. (Would more affluent veterans use private health care?). At a glance, I can't see results of any regressions on age but it might be worth contacting the authors about this.
How to adjust for this? One thing is just look at hospitalization risk (see AdamGleave's adjustment point (1)). However, it seems plausible that younger and healthier people would also recover better from less acute cases (and be less likely to have lingering symptoms). OTOH, there's anecdata and data (of less high quality IMO) suggesting that Long Covid doesn't fit the general patter of exponential increases in badness of Covid (and other similar diseases) with age. Overall, I'd still be inclined to make an adjustment of risk down if you are under 35 and healthy.
This is a good point, the demographics here are very skewed. I'm not too worried about it overstating risk, simply because the risk ended up looking not that high (at least after adjusting for hospitalization). I think at this point most of us have incurred more than 5 days of costs from COVID restrictions, so if that was really all the cost from COVID, I'd be pretty relaxed.
The gender skew could be an issue, e.g. chronic fatigue syndrome seems to occur at twice the rate in women than men.
My new estimate from the calculation is 3.0 to 11.7 quality-adjusted days lost to long-term sequelae, with my all-things-considered mean at 45.
I'm assuming the all-things-considered mean should be 4.5?
I did actually mean 45, in "all-things-considered" I was including uncertainty in whether my toy model was accurate. Since it's a right-tailed distribution, my model can underestimate the true amount a lot more than it can overestimate it.
For what it's worth, my all-things-considered view for Delta is now more like 30, as I've not really seen anything all that compelling for long COVID being much worse than in the model. I'm not sure about Omicron; it seems to be less virulent, but also more vaccine escape. Somewhere in the 15-90 day range sounds right to me, I've not thought enough to pin it down precisely.
Here's another BOTEC, by Matt Bell:
These numbers are low, but not low enough to ignore. Earlier we decided that the quality of life hit from long COVID after a non-hospitalized acute case was 18%. If you’re a 35 year old woman, and your risk of ending up with lifelong long COVID from catching COVID is 2.8%, then catching COVID would be the same, statistically speaking, as losing (50 years * 0.18 * 0.028 * 365 days/year) = ~90 days of your life. Ouch.
We can also look at just the "worst case scenario" – catching long COVID that doesn't go away for years AND limits daily activities a lot. This number feels a bit more like a "mortality" rate – except in this case you don't actually die, but your life is forever altered, and you can't hold down a job anymore or do most of the things you used to love to do.
A 35 year old woman runs about an 0.5% chance of the "worst case scenario" outcome if she gets Delta. For comparison, 0.5% is about 42x your chance of dying in a car crash in the next year.
I think the main differences are using studies with higher excess burdens and using a lower reduction factor to translate to lifelong risk. On the latter:
In the end we need to make an educated guess, even if it's a low-confidence one, as to how often long COVID that lasts 4.5 months ends up being lifelong. Based on the SARS data, we could guess that 80% of hospitalized acute COVID patients that that have long COVID at 4.5 months end up having it for the rest of their life. Patients with milder COVID cases tend to get less physiological damage during acute infection, so it's possible they'll have higher recovery rates. Again taking an educated guess and going on even less data, we might expect that 50% of long COVID cases for mild acute patients at 4.5 months end up being lifelong.
I'm pretty confused about how PCR testing can be so bad. Do you have more models/info here you can share?
In particular, I think it might be the case that we've done something like overupdate on poorly-done early Chinese PCR. When I looked for data a while back, I only found the early Wuhan stuff, and the company-backed studies claiming 98% or 99% accuracy, neither of which seem trustworthy...
I currently suspect that PCR tests are effective, at least if the patient has grown enough virus to soon be infectious. I'd like to know if this is true. The main beliefs I have here (that may well be false):
- The PCR methodology, when done right, should detect the presence of tiny amounts of viral fragments.
- The amount of virus needed per unit saliva to infect someone is at least a few orders of magnitude less than the detection threshold for PCR or other amplification techniques.
If my picture is right, I can perhaps still believe in a 50% false negative rate, but I would look to explain that as "you tested them too early in the infection", and would suspect the false negative rate to be more like 1-5% for a patient that's shedding enough virus to be infectious.
In short, I don't really know how it can be as bad as I claim it is. It seems like it should straightforwardly be highly accurate because of your two points: the sensitivity should be at a much lower threshold than the amount needed to infect someone.
Yet, I still believe this. Part of this belief is predicated on the heterogeneous results from studies, which make me think that "default" conditions lead to lots of false negatives and later studies showed much lower false negatives because they adjusted conditions to be more sanitary and less realistic. However, this is just an extrapolation, and I haven't looked into these studies unfortunately.
The bigger reason for my belief is that I've seen several people almost-definitely get COVID and then test negative.
- First data point: B was out in public unmasked, got it, then their family got it. 4ish people showed symptoms, one didn't. 2-3 tested positive for COVID, the others didn't, including 2 who tested negative 3ish times in a row, using PCR. B was one that tested negative repeatedly. B was notified shortly afterward that the person they were with in public had tested positive for COVID.
- Second data point: C went back to university in Sep 2020. Two of their family members visited. Shortly after, C got pretty sick and tested positive for COVID. Then their family members got pretty sick with flu-like symptoms. Both family members went to the doctor after symptoms and tested negative by PCR.
- Less-strong data point: D flew on a plane from the Bay late-Feb/early-Mar 2020. They landed, two days later they got sick with cough, maybe more, felt pretty bad, and had an spO2 of 85. They tested negative 2-3 times by PCR.
I heard about these cases because they were fairly close to me. There were maybe 2 other cases as close to me as these, so these represent about half my epistemic exposure to COVID cases on the ground.
I don't know how to possibly parse the first two cases aside from saying that the PCR tests gave false negatives. You can't even say "they got the flu"—their family members tested positive for COVID! The best alternative explanations seem truly terrible: there's a minuscule chance I just got a wildly skewed sample, or I could've done a truly abysmal job at noticing some selection effect. So I feel like I basically have to take them at face value. Using data points 1 and 2 only, and adding 2 positives from cases close ot me, the PCR tests gave false negatives roughly 7/10 + 2/3 + 0/2= 9/15 times, ~60% false negative rate, but if you count by person this is only ~40% false negative, which is the better way to look at it to correct for the selection effect of people who test negative getting tested repeatedly. Maybe someone got rapid when they thought they got PCR, and you lower 10% to bring the total to 30% false negative rate by person. But not much more in sample mean.
As I said, I don't really understand how PCR tests can be this bad. However, it would tie up very neatly if, for example, COVID just didn't make it to the nose in a lot of patients. Perhaps orders of magnitude more is coughed out of the lungs as an infection vector than is exuded from the nostrils. Or perhaps the efficacy of swabbing varies a ton, or the efficacy of testing—a lot of the negative tests I know about were from Red states, and I can't help but wonder if the old "getting the results you want to get" effect is striking in another wild circumstance (but of course note the selection bias since I know of more cases in Red states).
And even without knowing how PCR tests can be so bad, I don't feel like I'm going that much out on a limb when I'm imagining there might be lots of heterogeneity in how they're done. Even if the best tests are really quite good, if the worse tests have user-error rates of one in five, and these are selected to be the ones more in use where COVID outbreaks are (due to culture or the obvious causality), you could potentially have a lot of people with 20% FNR rates. Also, while I think lots of criticisms that "the lab is different than the real world" are misplaced, COVID tests seem like almost a central case where you'd expect that specific failure mode.
(I probably won't delve into the papers to try to figure this out, but I would love to hear from anyone else who might have alternative hypotheses about this, or reasons why "COVID doesn't always go to the nose" shouldn't be the default hypothesis here.)
Another cool data point! I found a paper from Singapore, Jul 2020, testing tear swabs but incidentally giving a bunch of PCR tests too. I'm much more likely to trust a paper that gives PCR tests incidentally, rather than is directly testing their effectiveness with researcher bias toward better results. By counting up the squares by hand, this paper shows 24/108 PCR tests came back negative if I counted correctly: that's 22% false negative rate (FNR).
Now, for adjustments:
- First, these patients were recruited from a hospital. So they obviously have much higher viral load than the average person, so we'd expect higher FNR for the general population. (And we see the expected relationship between viral load and positive results: people with average low Ct values (meaning high viral load) rarely test negative, but those testing negative lots have very high Ct on their positive tests.)
- On the other hand, only 2/17 patients test negative >50% of the time; a lot of the negatives come near the end of a patient's sickness or hospital stay. So we don't see great empirical evidence for the hypothesis that some people are consistent false-negatives. If you take out the negatives-at-the-end effect, there are far fewer false negatives, maybe 5-10%. However, this is basically moot because of the selection effect for the hospitalized as mentioned above. Of course you'll see hardly any consistent-false-negative-patients in the hospitalized!—the fact you see any macroscopic number of false negatives in the middle of progression is a terrible sign (and, if there were any fully-false-negative patients, we wouldn't see them anyways! Bad filter).
- And we do see the requisite theoretical evidence. Because of the two patients with repeated false negatives and low viral load when positive, we can easily extrapolate that some patients just have slightly lower viral load and test negative consistently.
Overall, there isn't much easy way to convert this study into "FNR on asymptomatic individuals who get tested". However, I think if 5-10% of tests on the hospitalized came back negative, that strongly implies more than a 20% FNR on the asymptomatic. I would personally guess that this lends credence toward 10-40% FNR on the symptomatic and 20-80% FNR on asymptomatic. (Lest I double-count evidence, let it be known I'm basing these numbers in part on the above analysis of my personally-known symptomatic individuals with ~40% FNR.)
Also, when I look around, I find charts like these that suggest the claimed false negative rates vary absurdly!
I believe FNR depends on swabbing (which varies based on equipment and individual doing it), on PCR equipment, and on the patients (e.g. how early are you testing people? age of patients, etc). Then there's issue of how you get ground-truth which might also contribute to variation in these estimates.
I once looked into the effectiveness of Australia and New Zealand's quarantine programs to get a sense for this. I think, until recently, basically no infectious cases made it through their 2-week quarantines. Their track records have become more marred since Delta arrived.
For New Zealand, if I recall correctly, basically no community infection clusters were due to quarantine breakthroughs (citation needed!). Of the cases caught with PCR, 80% tested positive on day 3 of quarantine, and the remaining 20% were positive on day 12.
So while some cases might have not been detected, it seems like these didn't go on to infect others after the 2 weeks. The people getting tested on day 3 could have been infected on their plane flight, or perhaps some days before their flight. I'd guess the median infection was a week old by day 3 of quarantine.
Anyhow, NZ's numbers seemed to rule out a 50% false-negative rate, because I think their quarantine would have failed if so. They also seem to rule out a 2% false-negative rate, at least for tests done early after infection.
I think, until recently, basically no infectious cases made it through [Australia and New Zealand's] 2-week quarantines.
In Australia, hotel quarantine has caused one outbreak per 204 infected travellers. Purpose-built facilities are far better, but we only have one (Howard Springs, near Darwin) and the federal government has to date refused to build any more.
Our current Delta outbreaks are tracable to a limo driver who was not - nor required to be - vaccinated or even masked while transferring travellers from their flight to hotel quarantine.
The main source of our success has been in massively cuts to the number of travellers we allow in, and that has it's own obvious problems...
In Australia, hotel quarantine has caused one outbreak per 204 infected travellers. Purpose-built facilities are far better, but we only have one (Howard Springs, near Darwin) and the federal government has to date refused to build any more.
But no cases of infections slipping thru the testing, no?
As far as I know none of our leaks have been by releasing an infectious person after a negative test result.
It's possible for PCR tests to return negative for a very early (low viral load) infection though; that's why for high-risk travellers we do PCR tests on days -3, 1, 5, 11, and 14 of the quarantine period. For low-risk settings, ie contact tracing, you only need to isolate until you get a negative PCR test result.
Curated. It seems really important to figure out how to deal with the Delta variant, and orienting around how to interface with covid variants longterm. I appreciated this post for:
- Striking a good balance of presenting current facts on the ground
- Noting where assumptions lie
- Being clear about it's epistemic status, while
- Aiming to be useful in the immediate term.
I didn't read the Lancet paper. Are they able to rule out selection biases? It's possible that people who got mild Covid will score slightly lower on the cognitive tests (even if you adjust for observable demographic differences). It also seems plausible that this very small measured difference (for non-respiratory Covid) will further diminish over time. (Also the mean age is ~47 and so a 30yo should expect smaller effects and better recovery in any case).
It's not a great study. It's observational, and the best it can show is correlation. It's just as likely that people with that score lower on cognitive tests are more likely to get covid, which doesn't seem unrealistic. A bit of armchair analysis (with only a little bit of ax-grinding): https://sebastianrushworth.com/2021/07/26/does-covid-cause-brain-damage/
I edited the section to include some more thoughts on the paper's quality. In brief, I expect a 2x diminishment over time (though this was conservative and it could easily be larger); I expect the selection bias is definitely real, though there's a countervailing effect from most of the COVID cases being self-reports which I expect means higher conscientiousness and many missed cases from the group otherwise being selected for; I also think the ventilator impairment matches well with other evidence we have about ventilator impairment and is probably not a large overestimate, though this may not shed much light on the smaller disease burden impairment integrity; and I hadn't noticed the age difference, thanks a lot for pointing that out!
Rapid tests are even worse, <50% sensitivity unless they’ve gotten much better in the last 6 months
Is that for professionally administered or self-administered tests? Since picking up a free box of the self-tests, I'm sceptical that they're worth anything. I've never yet managed to take a swab of my throat (the muscles just go >!erk!< when the swab gets near), and I don't know if I've ever worked it up my nose as far as it's supposed to go.
The test does not distinguish between someone who is virus-free and someone who didn't properly swab. I checked this by doing a test without any swabbing, and it came up negative, not void.
In roughly Feb/Mar/Apr of 2020 I had an unexamined belief that the cause of the failure of the US medical system and government to protect the health of the citizens was that "no one knew that it was feasible to do mass public testing for relatively little money" and so I poured brain power into designing a mass testing framework to test >1 time per week "roughly everyone who wanted to be tested".
The process that seemed likely to work best didn't use swabs. I was worried about this, because it seemed necessary to have a thing simple enough for almost anyone to do right... but then research came out that suggested that saliva and gargling based sampling methods actually beat "nurses sticking swabs up your nose with skill". So that was nice for a while. (I haven't gone back to double check this, it might be wrong at this point? Maybe?)
The idea was just to do saliva-and-gargling, and you got a cup in the mail, and swished, and let the pickup person take it away for pooled analysis.
I think I could get the price down to roughly "cost of the moving the cups to the home then to a lab + chemistry cost of approximately LOG(cups) test reactions". So like... $0.50 per cup? So like $150 million per week to test 98% of anyone in the U.S. who wants a test, every week?
This seems like part of how one would eradicate covid. After you eradicate you still have to keep the green zone (that is: the whole country) green before you can dial back the testing? Success: technically possible!
Only about five or six regulations had to be ignored to make it work (like OSHA and HIPAA stuff and so on), so it seemed promising.
Surely a technical solution could be found to the social problem of dumb regulations and a big fat cone of sound absorbing mud in the place where "people working to solve coordination problems" should be?
In the happy model: covid could maybe even be a retrospective blessing for helping to demonstrate that problem solving is possible! Fixing not just covid, but also cleaning up some of the mud!
But I could never communicate the relevant vision to the relevant people (do "the relevant people" even exist in America anymore?) and then I ran out of steam. Convincing medical workers to break laws doesn't work. Convincing mayors runs into state/federal prosecution. Governors are hard to talk to, and harder to convince to give blanket pardons for violation of OSHA/HIPAA/FDA rules. Each agency would have to change their rules in a coordinated way to allow cheap mass testing to work... The higher you go the less they understand and the more they want certainty that they won't be yelled at by dumb voters. The lower you go the more you find scared little rabbits in tiny little cages. The rabbits can't promise the wimpy governors that "the thing will work...or if it doesn't work we won't blame you" because they're just rabbits :-(
In concurrent computing, a deadlock is a state in which each member of a group waits for another member, including itself, to take action...
That's us! That's America!
So for a while I switched AWAY FROM the unexamined belief that "no one knew that it was feasible to do mass public testing for relatively little money" TO an unexamined belief that America was semi-intentionally self-destructively stupid, and so covid and covid's tragedies were our just and balanced nemesis, acting upon our collective wrongness, in a tragically balanced way.
This oscillated between "collective (because officially democracy)" and "just the elites deserve the suffering (because de facto oligarchy)". Now I just feel sad. Even suicidally stupid people don't deserve to die :-(
But... ignoring the big picture insanities of my unexamined past beliefs... I DO think that saliva tests are probably Correct (and pooled testing could be very efficient (even though it is still basically illegal)).
That's a really interesting point. I wish we had a conventional way of suspending certain regulations in certain circumstances, rather than having to wait decades for an entirely overhauled piece of legislation on the whole macrotopic. Are there things like, I don't know, executive non-enforcement orders that ever get used similarly?
Do you know of any non-pooled tests that are cheap and fast, that perhaps a group of individuals could order loads of? I've heard people talk about LAMP and such for a while but without any persuasive end-to-end evidence.
A complexity here is that "the macrotopic to reform" is "the entire system... like practically all of it".
You can't just delete the FDA because while the FDA is the current lynchpin (and "FDA delenda est"), to have a good system, that was not broken (in the way or other ways), you also need tort reform, and insurance reform, and on and on and on...
As near as I have been able to tell, the reason Pooled Testing is illegal, is because there are OSHA laws protecting chemists, basically, so if a chemist in a lab is running test reactions all day every day, and they accidentally put a "cut finger" into a jar of slime... OSHA requires that the jar of slime be traceable all the way back to someone who can be tricked or compelled or begged into somehow taking an HIV test. If the jar "from the patient to the chemist" lacks a chain of custody, then the chemist's employer is not allowed to have the chemist to do that job.
If you have a cup, and can check the box "[X] let me know by text message" and write your phone # on the cup, and your cup goes into a jar with 2000 other people, and the whole 2000-cup batch gets a negative, then 1 test reaction generates 2000 "negative results"... great... except OSHA will shit a brick. That's not allowed.
OSHA things the poor chemist can't just refuse? Isn't grownup enough to decide for themselves what risks they'll take for what amount of pay?
The macrotopic for which we need to "suspend certain regulations", as near as I can tell is simply: "using paternalistic reasoning to forbid people from owning risk to themselves after looking at local factors and deciding for themselves".
Delete all ALL the laws whose only flimsy excuse is paternalism, and then I think any small city mayor could save their small city from covid using stuff you can buy at the grocery store (or maybe a lab supply company) based on ideas and tutorials you can look up online... Then they hire some (potentially inadequate) people to take some risks and learn on the job, and it "solves for covid"? I think?
(Though maybe a non-assertive half-competent chemist or two gets HIV. In my mind: worth it!)
Do you know of any non-pooled tests that are cheap and fast, that perhaps a group of individuals could order loads of? I’ve heard people talk about LAMP and such for a while but without any persuasive end-to-end evidence.
Antigen tests. They take 15min to give results, and are 0.8€(retail) here.
Hm, I meant to exclude those because of their abysmal sensitivities, but I suppose I should revisit them now in case they've gotten better.
They are not that bad.
sensitivity (Ct ≤33): 97,1% (132/136), (95% CI: 92,7%～98,9%)
sensitivity (Ct ≤37): 91,4% (139/152), (95% CI: 85,9%～94,9%)
Considering the price and simplicity they are often worthwhile.
We do have some evidence about which world we’re in. There are studies which find pretty big differences in level of antibody titer produced by the vaccinated, and in some cases where they have almost no antibodies it’s pretty clear that this means immune responsiveness is going to be at fault when they get sick. And I think there are studies finding correlation between titer and effectiveness. Both of these point toward innateness. But we also know that it has to be true that for many of those with low levels of antibodies, a larger dose will push them over the edge. There is also slight evidence from the Israel numbers, which give effectivenesses that vary some over time, that there’s a serious behavioral/environmental component.
So, we're in both worlds. VE is a function both of immune response and viral load exposure. Which one is relatively dominant may be important for behavioral implications (I agree with you!), but this doesn't have to be an either or. "Breakthrough" cases can have multiple input factors. Even the "innate" world comes with the question of whether the vaccine stochastically increases titers across the board or stochastically increases titers only among a susceptible type of person (is it a single distribution or a mixture distribution?). But once we think we're living in a world where both matter (actually, I wonder to what extent this community endorses this POV or if generally LW thinks it is an either or situation?), and once we obtain a ton more info, the behavioral recommendations can be either really complicated and theoretically optimal but impossible to follow, or they can be simpler and sub-optimal but implementable. We see this with vaccinateds masking - the distribution of titers pretty much indicates vaccinateds will take care of wild type just fine, so the CDC cuts the mask recommendations (despite there being some variation). The titers vs. Delta are not quite so great, so the CDC re-implements the mask recommendations (despite there being some variation). The world is messy.
Yeah sorry, I meant "where on this axis" are we—I had meant the evidence to show "we're not in one or the other", but I'll edit for clarity. (And obviously it's much more complicated than just this axis, as you point out. I was just trying to make it understandable.)
I agree with the recommendations having the possibility to be complicated+optimal XOR implementable and think it's an important and somewhat underappreciated point. It makes it somewhat more complicated to give policy recs to family/friends but I'm glad I'm slowly learning how to give rules that are more implementable.
Again, remember that Delta might top out in a few weeks anyways.
In the UK (which had a Delta spike), COVID cases have fallen every day for a week.
... and then, after falling substantially but not nearly to pre-Delta levels, the fall has levelled off and the case numbers may perhaps be rising again.
(I don't know what to make of this. Burning through some particularly-exposed subpopulation and then more slowly getting to everyone else? Some weird transient effect from the end of the school term?)
See also: Twitter thread looking at hospitalization/death data and concluding that the big fall was mostly not real because by now we should be seeing it in those data.
A couple of things that seem possibly missing from the reasoning there: 1. if time-to-hospitalization or time-to-death is more variable than time-to-testing-positive, then those later but more reliable indicators will be low-pass-filtered relative to the case numbers, which would mean that looking at early hospitalization/death numbers might make even a real decline look fake or exaggerated; 2. if the rise is all about Delta taking over and Delta is more harmful than the previously dominant strain, and if during the "falling" period the proportion of Delta in the population relative to other strains is still increasing, then that too could make the hospitalization/death numbers not fall as dramatically as the case numbers even if the latter are genuinely falling rapidly. BUT I am a very long way from being an expert, and don't know whether the actual numbers are such as to make either of those a real issue.
(I think #2 is not a thing; almost all UK cases were Delta too early for that to be the case. But, again, not an expert.)
while different studies makes this look more or less difficult, India and the UK both appear to have successfully done this without obviously draconian behavioral interventions
India has largely been successful after they gave everybody Ivermectin.
The UK data looks very weird given that the drop in cases correlates with them releasing the lockdown. Understanding what happened there should be a pretty high priority.
Rapid tests are even worse, <50% sensitivity unless they’ve gotten much better in the last 6 months
In Berlin, rapid tests are available at many locations onsite and every citizen can get one per day and then gets the result within 15 minutes per email. The one that's used next to my place has sensitivity: 98% and specificity: 99,10%
Principle: Variolation maybe good, still hard
Given that RaDVaC is an available option it's very unclear to me why anyone would do variolation.
The UK was already mostly open before July 19. So "release the lockdown" is highly misleading. It's also too early to see the effect of July 19 on cases.
Thanks for sharing this stuff. I like the principles, and I like the bottom line being a rate of exchange: 1 hour of your life lost every 1k-5k uCOVIDs. Will go do my own calculations to figure out what that implies for me...
I'm concerned about nonlinearly bad outcomes from losing a few IQ points. I don't think it's just that if I lost 5 IQ, I'd work more slowly. Perhaps I just wouldn't understand certain things; there maybe thoughts I can no longer think; insights I could no longer have. I'm worried about this both for impact reasons (helping solve alignment) and for personal reasons: my mental acuity is important to my life-enjoyment, via grasping new concepts and enjoying self-study (you do mention happiness effects). On an extreme end of the spectrum (which COVID would not take me to), I doubt an 80-IQ TurnTrout would much enjoy self-study, however slowly that progressed for him.
I'm similarly worried about about "tipping point" effects from other possible long COVID symptoms, like fatigue ("straw that broke the camel's back"; fatigue may not just reduce my productivity and enjoyment by 5%, but make it consistently hard for me to do important thing X at all due to insufficient activation energy).
IQ affects many things. If you roughly double your income when you go up 60 IQ points, each IQ point is about 2% added income. Each IQ point is probably also .1% happiness (.01/10). It also probably reflects some underlying worse health that may have other affects. It’s hard not to overcount when something is correlated with everything. However, I think the largest effect here will be the money-equivalent (impact, perhaps). We can double this at the end to account for other effects. Giving up 1/50 of this for an IQ point is equivalent to a lost week of production every year, which, while not the same as lost life, is still pretty bad.
What does "this" refer to, above?
Right. I think these are all fair, and I've tried to take them into account and be pretty conservative in not underestimating the risk. There are obviously a bunch of balancing forces on the opposite side from those you've laid out—eg the tipping point effect balances against the "you never notice" effect, otherwise you're double-counting. In general, I'm trying to find a coherent picture that takes into account the negatives while not overcounting them—it seems very easy to overcount given that all good things correlate and all bad things correlate, but small affects don't actually send one into a spiral.
To sanity-check my calculations, let's consider what it would do to increase the amount of weighting to IQ by a factor of 3. I had estimated a .15-point loss in IQ is .5% of your life-equivalent lost, so tripling this would mean 1.5%. Then losing 1 IQ point would be equivalent to 10% of your life, and every fifth of a standard deviation would be a third of your life-equivalent. This would mean at your margins, exercising like two half-hour amounts in a week would cause your life to get 20% better. I think that stretches the bounds of credulity. Even a factor of 2 increase seems to stretch things, unless you're very comfortable counting 70% of your life's value as coming from that small edge of intelligence that differentiates eg a better-than-average grad student from an average grad student (maybe IQ 140 vs IQ 130).
Re "this", I think I meant productivity but had changed the sentence. I'll edit to fix.
I would guess Long Covid then reduces a 30yo’s average productivity by on average .3%.
This seems like a vast underestimate of the impact of long covid; the long-term loss of energy that people with post viral fatigue suffer is way worse than that.
From Anne's document:
I have Myalgic Encephalomyelitis / Chronic Fatigue Syndrome (ME/CFS). Many Long COVID sufferers have what seems to be ME/CFS. It’s a bad illness. You don’t want it. Nearly every day of my life since I’ve developed ME/CFS has been centered around trying to find some way to get better. I think people without ME/CFS undervalue how bad it is because it’s hard for them to fathom that anything this bad (yet non-fatal!) even exists.
The long-term post viral fatigue/CFS/ME outcome is probably more like a 50%-70% loss in productivity than the 0.3% estimated here; think of people giving up their career, losing relationships, giving up activities and social life due to a permanent illness, etc.
Outcomes this bad are probably less than 5% of cases, Anne threw around the 1/3 of long covid cases figure, but we don't know at the moment.
I'm fairly sure that the .3% was averaged across the 5% if people reporting long term symptoms. The vast majority will be mild, while a small fraction will really suck (I think, given the model of the post)
Actually, this is averaged over all 30yos who get COVID—I realize this was unclear as a summary, I'll fix. So it's equivalent to about .6% of them getting horrific CFS and losing half their life-equivalent. (Obv in reality you're looking at a smooth distribution of badness).
I think this is reasonably supported by my own experience of seeing ~10 30yos getting COVID pre-vaccine and not having any CFS, and now we have 3x less risk with the vaccine.
if interval is 4 instead of 5.5 days, this would mean that reported R of 7 would turn into R of 9^(4 / 5.5) = 4.
I think this 9 should be a 7?
Here is another study on viral load:
"A recent study has even suggested that respiratory samples from people infected with the Delta variant can harbor up to 1000 times higher viral loads compared to samples with variants that are more closely related to the original Wuhan strain, although the sample size of this study was very limited. Here, we have compared the viral load in 16,185 samples that were obtained during which non-VOC, the Alpha (B.1.1.7) or Delta variant (B.1.617.2) were dominant as evidenced by genomic surveillance. We found that the Delta variant contained about 4-fold higher viral loads compared to the non-VOC or Alpha variants."
But also see the comment there.
Thanks for writing this!
It seems like the main thrust of this is to compute individual average risk and construct policy recommendations from there. I have two main objections to this approach.
- My utility might be closer to log in quality-adjusted life-days. For example, I think I would much prefer to lose 1/100th of my remaining life rather than take a 1/100 chance of immediate death. Similarly, it's not clear to me that a 100% chance of a 50% loss of productivity is like a 50% chance of a 100% loss. This makes your averages seem to gloss over important distinctions.
- I'm not sure the individualistic approach to policy recommendation makes sense. For example, suppose vaccines are exactly 90% effective. This isn't enough for me to make big policy changes (EG, if going to a party felt unsafe before, it still feels unsafe after). However, me and my entire extended group of friends getting vaccinated makes a huge difference (the hypothetical party could now feel safe).
So I don't feel like your average-case cost is that relevant to whether the rationalist community should take specific precautions. The cost of a specific precaution scales linearly with the number of people taking that precaution, but the benefit scales much faster, for a community of interacting people.
I'm left feeling like you probably ended up underestimating the level of caution we should have.
Since almost all of these costs are from Long COVID, I think these are actually more like a constant 1/100th of remaining life than 1/100th chance of immediate death. However, since 5% of the cost is from death and another decent chunk is from possibility of CFS, I would understand if you made a small adjustment here. Personally, I don't think I'm going to increase the estimate of my own risk, though part of that is because I think I was conservative enough that I'd be skewing things if I made even more implicit adjustments toward higher risk.
Yeah, sorry about the confusion about community policy vs individual. I originally tried to give community advice but it got too complicated too quickly, so I just shipped with the individual policy estimate. My ideal way for people to interpret this is something like the precursors for a dividing line between the attractor states of "just act like normal and stay away from your cautious friends" and "just keep being cautious and stay away from all the risky people". E.g. I think there should be bubbles of people who try not to get COVID and collectively take on less risk than individually optimal, but other people like myself should just take individually-relevant levels of risk and live the next few months in contact with others like ourselves. Ideally I will have a less-simplified version of this written up soon, but that's definitely questionable.
Curated and popular this week