Onni

On alternative terminology: It feels more natural to me to think about it in terms of the inverse, of how much oversight and how carefully-designed controls / checks and balances you have. Also importantly, how much centralization into "one" AI, vs. diverse competing AIs. As you have more powerful AI, to some extent it becomes by default more difficult to oversee, and in a competitive situation there's a temptation to skimp on those checks and balances. This feels closer to how we think about it in the human case, e.g. there's a temptation to remove check... (read more)

Some kind of meta grumbling about this concept:

Both trust handoff and decision handoff feel a bit slippery and “unfalisifiable” as concepts. What does it mean to automate the economy in a way that isn’t one or both? You say that ideally you’d want to do “trust handoff never thanks to some fancy scheme for AIs to watchdog each other”. But then your description of the “other extreme” of trust handoff sounds a lot like just such a fancy scheme. So is that trust handoff or not? I get that it’s a spectrum, but it seems really unhelpful to collapse this into a b... (read more)

I really loved reading this, it resonated quite deeply. I've felt much the same, though for me the biggest thing that made me misanthropic was reading about EA and all the implied ways that most people are arguably monstrous by omission and conformity. After that, reading Nietzsche didn't make it much worse.

In recent years I've been feeling somewhat less misanthropic, so wanted to quickly say a bit about how that's happened, in case it's useful. Probably the biggest influence was reading Joseph Henrich's The Secret of Our Success. The core thesis of the bo... (read more)