This is a special post for quick takes by Fergus Fettes. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

New to LessWrong?

1 comment, sorted by Click to highlight new comments since: Today at 9:39 AM

I wanted to make a comment on this post, but now I'm not sure if it is supported. The comment follows:

--

Great post! One point:

And that is exactly what'd we necessarily expect to see in the historical record if mesaoptimization inner misalignment was a common failure mode: intelligent dinosaurs that suddenly went extinct, ruins of proto pachyderm cities, the traces of long forgotten underwater cetacean atlantis, etc.

There are a few circumstances under which we would expect to see some amount of archeological civs, such as:

  • transition to writing being unlikely (oral-culture civs should be common)
  • industrialization being unlikely (pre-industrial civs should be common)

Ah but wait, we do see those (they happen to be the same species as us). Actually there have been a few civs that have gone under in a meaningful way (maybe the Mongol or the Khmer Empires would be relevant examples?).

I do agree with the view that, as humans, our total score seems very robust against local variation (local mesaoptimization misalignment). But it doesn't seem like that isn't a thing that can happen, just that we have had enough variance as a species that we have survived.

In the Atomic Age this seems less obviously the case. Inasmuch as we are one global civilization (or we have civs powerful enough to act like it), it seems possible to suffer from catastrophic mesaoptimization misalignment in a way that was not possible before.

I think this is a slightly different argument than some of those stated below, because it looks at the historical record for examples of inner misalignment rather than strictly trying to predict global doom in the future. At least I think it addresses the claim in this specific paragraph.
 

--

Have any civs really fallen catastrophically, that wasn't directly attributable to plague? Not really right? I remember some 80k research about it. Cities are also famously robust, almost immortal. Some good examples would be needed for this point to stand.