As Covid is for the most part over and tests have not been in short supply for a very long time now, eliminating most of the rationale for complicated testing procedures meant to optimize test count/cost, it would be interesting for anyone who was involved in this to do a retrospective or post-mortem. Was pool testing useful? Why or why not? Can it be improved for future pandemics and should we care about it?


My impression, not having paid much attention to the details at the time (AI scaling was more important), is that various group/pool testing approaches were useful in a few niches, but were rendered useless in many places due to politics/organization/social flaws, and didn't apply to most places.

  1. I read that use of pool testing was common in China early on in 2020 as part of the hotspot strategy to economize on testing extremely large groups of people where only a few would have active Covid. This is the most logical place to apply it, and it was a major government & societal priority, so that is a success.

    I get the impression that they were able to switch to individual testing fairly early, in order to get faster turnaround and do more localized testing. But to the extent that pool testing was effective from the start, before testing manufacturing could scale up, and helped squash Covid when it was still squashable, then it was very valuable even if obsoleted.

    Of course, this evaluation can change depending on how Covid ultimately shakes out in China. Was it part of a heavyhanded but ultimately highly effective Covid Zero policy which led to it turning into a hermit country enduring devastating economic fallout while merely postponed the inevitable variant which could defeat Covid Zero? In which case, pool testing was a tool which played a part in buying China entire years to vaccinate but its good was rendered useless when they bizarrely failed to make any good use of the time, dabbling in ineffective vaccines etc, and in practice, only doubling down repeatedly on coercive testing and surveillance until the martingale blew up in their face. Or was it part of a visionary resolute effort which spared China the mass death of the rest of the world as it came out the other side by persisting so long in country-wide lockdown until Covid just sorta stopped being a problem somehow? Then its good was unalloyed and it's sad so few other countries could boast a better record (eg Taiwan; I dunno if they made any use of it).

  2. Sewage/water testing in places like colleges: various universities or towns made heavy use of what is in effect pool testing to do mass screening of water to detect infections and then hotspot them. This was only useful as long as you have a baseline of ~0, otherwise testing water merely gives you a proxy for overall infections, which is a lot less useful than using it to detect & squash hotspots. In individual cases of orgs with great discipline, this apparently worked well, and avoided the need for constant individual testing (which was expensive, not possible early on, and still inadequate - as all the infections inside 'bubbles' show, just too much leakage, test error, incubation, idiosyncrasies and so on).

    Since most of these places were in countries where Covid became endemic eventually anyway, and everyone got infected, and in cases like colleges the young students were the last to get vaccinated (because they are the least harmed) and so needed to buy a lot of time to avoid infection before vaccination (which probably didn't happen for most of them), these success stories seem to boil down to just buying time, which is a good deal less impressive.

At the margin, doesn't seem like improvements to pool testing would have helped much compared to other things like faster vaccination approval.

73

New Answer
Ask Related Question
New Comment

1 Related Questions

Parent Question
4 comments, sorted by Click to highlight new comments since: Today at 8:32 AM

Was it part of a heavyhanded but ultimately highly effective Covid Zero policy which led to it turning into a hermit country enduring devastating economic fallout while merely postponed the inevitable variant which could defeat Covid Zero?

Strong vote in favor of this interpretation of China's policies. Doesn't give me much to say about the post-mortem, though.

I'm catching up on Zvi's COVID link roundups and it looks like, very unfortunately, the pessimistic interpretation has turned out to be right. What was Xi's big COVID Zero exit strategy, what was his end game - how did China get from here to there when they refused to buy the best vaccines or ensure universal vaccination or stockpile Paxlovid or stagger infections by variolation or bulk up the healthcare system or... [many things which are more useful than security theater like spraying the streets with bleach or killing people's pets]? How do you stop riding the tiger?* Turns out, there was no exit strategy. And now it's moot.

Pro-China advocates (Naomi Wu's tweets kept getting recommended to me, for some reason, although I doubt she was particularly notable or extreme so shouldn't be singled out) downplayed the doomer forecasts that Omicron was eventually going to eat through 'COVID Zero' & praised its success. Doomers forecast that the longer 'COVID Zero' was enforced, the closer China would come to a breaking point, that Hong Kong was the harbinger of what was to come, and the results might be disastrous. It hadn't happened yet, and might not for a long time, but with no exit strategy, the results would be real bad Real Soon Now™.

Aaaaaand... that appears to be what has happened last month. They could have ended COVID Zero at a time and place of their choosing, after having prepared for it; they chose 'end it never', which was not an option, and so it ended at a time not of their choosing. As the joke about bankruptcy (or any other exponential process) goes, how does it happen? "First slowly, then suddenly." COVID Zero failed slowly, then suddenly. There are some interesting details about the protests and suppression thereof and whatnot, but that's the summary: China suppressed COVID until it didn't, and then exponentials gonna exponential.

The CCP is falsifying statistics now so it's going to be hard to ever get certainty on how many deaths are happening. (Age-pyramid approaches won't work because deaths are concentrated in the elderly, and the use of excess all-cause mortality is sufficiently well known now that they may falsify that as well, so the old approaches to figuring out the real death toll from all the CCP coverups like the Cultural Revolution or Great Leap Forward won't work.) Nevertheless, the reporting on the ground plus the reasonably well understood COVID death rate mechanics implies things are bad. And there is no going back to COVID Zero after they have "let'er rip" like this. (And then there's the reinfections! Fun.) It's not clear that China is going to wind up with all that noticeably better a report card than many countries (although I'm sure they are already working hard to argue that it was better than the USA's).

It's a fascinating, if horrific, example of the incompetence and perversities of a totalitarian dictatorship: despite the most intense panopticon in human history, they still suffer from the same internal information-making and public choice pathologies as the others, and decision-making is brittle and in the long run, incompetent. "What if Xi Jinping [or Putin...] just isn't that competent?" (This also connects to the CCP response to the chip embargo, which was first to handwave it away, then to announce a huge bailout, and then to... never mind, just cancel it and concede defeat?) Fukuyama undefeated - liberal democracy remains the worst system in the world, and the USA the worst country in the world, except for all the others that come up from time to time.

So, group testing failed in the sense that most of its value was contingent on enabling China to ride the tiger and figure out a way to get off it safely; but it didn't. Ex ante, a good idea; ex post, a much less good idea.

My summary: group testing is a good technical quickfix but one whose value was completely bottlenecked through squishier political/organizational limiting steps, and in the case of COVID-19, those limiting steps limited it to ~0 net impact, and it represents a cautionary lesson for technocrats.

* application to other x-risks like nuclear war or AI risk left as an exercise to the reader: yes, 'slow down AI capabilities progress' to buy time, great, yes, sure, I agree - but buy time to do what?

these success stories seem to boil down to just buying time, which is a good deal less impressive.

The counterpart to 'faster vaccination approval' is 'buying time' though. (Whether or not it ends up being well used, it is good at the time. The other reason to focus on it is - how much can you affect pool testing versus vaccination approval speed? Other stuff like improving statistical techniques might be easier for a lot of people than changing a specific organization.

How good would have been a pre-mortem as many products in tech get before they are released to the public?   https://medium.com/paypal-tech/pre-mortem-technically-working-backwards-1724eafbba02

New to LessWrong?