Gregory Lewis (Thrasymachus here on LW) posted a great summary of a bunch of arguments I've referenced many times, about how it's better to communicate resilience instead of imprecision to communicate if you are uncertain about something. 

Suppose you want to estimate some important X (e.g. risk of great power conflict this century, total compute in 2050). If your best guess for X is 0.37, but you're very uncertain, you still shouldn't replace it with an imprecise approximation (e.g. "roughly 0.4", "fairly unlikely"), as this removes information. It is better to offer your precise estimate, alongside some estimate of its resilience, either subjectively ("0.37, but if I thought about it for an hour I'd expect to go up or down by a factor of 2"), or objectively ("0.37, but I think the standard error for my guess to be ~0.1").

New Comment
1 comment, sorted by Click to highlight new comments since: Today at 10:40 PM

The rule in chemistry is that you can take measurements to one magnitude of precision beyond what’s printed on the device. The estimated digit adds accuracy without being misleading. If you say you have 0.37 mL, you’re saying you might have had 0.36 or 0.38 mL, but certainly not 0.35 or 0.39 mL.

You should determine a lower and upper bound to go with your point estimate, while taking into account that your equipment might be broken or that you’ve committed some other experimental error.

That parallels how we measure physical quantities in the physical sciences.

If you can’t establish a lower or upper bound for your estimate, that would be a critical fact to include. In that case, saying “my estimate might change by a factor of two if I thought for an hour” is analogous to adding too many significant digits. You‘re exaggerating the precision of your estimate.

I think that a point estimate unaccompanied by an estimated lower and upper bound - perhaps under some specific conditions (eg if AI doesn’t take off before then and there’s no other FOOM event), should be seen as not a measurement at all.

If you’re doing a Fermi estimate, I think ideally you would start by making an advance determination of the level of certainty you would need to get a meaningful result. When you’re done producing your estimates, check whether they’re capable of supplying you with the needed accuracy. If they aren’t, throw them out and choose a different project: your tools are too blunt for this knowledge engineering project.