In Things I learned Writing The Lockdown Post, Scott Alexander describes a really tricky issue when trying to quantify the effects of some policies:
This question was too multi-dimensional. As in, you could calculate everything right according to some model, and then someone else could say "but actually none of that matters, the real issue is X", and you would have a hard time proving it wasn't.
A long time ago, I remember being asked whether banning marijuana was good or bad. I spent a long time figuring out the side effects of marijuana, how addictive it was, how many people got pain relief from it, how many people were harmed by the War on Drugs, etc - and it turned out all of this was completely overwhelmed by the effects of deaths from intoxicated driving. If even a few people drove on marijuana and crashed and killed people, that erased all its gains; if even a few people used marijuana instead of alcohol and then drove while less intoxicated than they would have been otherwise, that erased all its losses. This was - "annoying" is exactly the right word - because what I (and everyone else) wanted was a story about how dangerous and addictive marijuana was vs. how many people were helped by its medical benefits, and none of that turned out to matter at all compared to some question about stoned driving vs. substituting-for-drunk-driving, which nobody started out caring about.
It might actually be even worse than that, because there was some hard-to-quantify chance that marijuana decreased IQ, and you could make an argument that if there was a 5% chance it decreased IQ by let's say 2 points across the 50% of the population who smoked pot the most, and you took studies about IQ vs. job success, criminality, etc, really seriously, then lowering the national IQ 1 point might have been more important than anything else. But this would be super-annoying, because the studies showing that it decreased IQ were weak (and you would have to rely on a sort of Pascal-type reasoning) and people reading something on the costs/benefits of marijuana definitely don't want to read something mildly politically incorrect trying to convince them that IQ is super important. And if there are twenty things like this, then all the actually interesting stuff people care about is less important than figuring out which of the twenty 5%-chance-it-matters things actually matters, and it's really tempting to just write it off or put it in a "Future Research Needed" section, but that could be the difference between your analysis being right vs. completely wrong and harmful.
The same was true here. How do we quantify the effect of Long COVID? Who knows? Given the giant pile of bodies, maybe we just round COVID off the the number of deaths it causes, and ignore this mysterious syndrome where we've only barely begun the work of proving it exists? But under certain assumptions, the total suffering caused by Long COVID is worse than the suffering caused by the acute disease, including all the deaths!
There is more, but this covers the phenomenon I’m curious about. Let me try to describe the problem in general terms:
Important policies have so many effects that it is near impossible to keep track of them all. In addition, some effects tend to dwarf all others, so it is critical to catch every last one. (Perhaps they follow a Paretian distribution?) It follows that any quantitative analysis of policy effects tends to be seriously flawed.
Do we already have a term for this problem? It reminds me of moral cluelessness as well as known and unknown unknowns, but none of those seem fit the bill exactly.