A few days ago, I published The Darwinian Honeymoon - why I am not as impressed by human progress as I used to be. To my gratification, it was quite well-received on Twitter, Substack and LessWrong.^[1]However, in subsequent conversations I realized that I did not communicate my core point well enough, given how abstract it is. So I wanted to write a short (and somewhat sloppy and galaxy-brained) post explaining what I mean a bit more.

Here’s what I am NOT saying:

“Humans will face the same fate as chickens”: Obviously, humans are much more powerful than chickens were, and things will go very differently.
“Humans will definitely get outcompeted by AIs”: I’m not making a direct claim like that, and I wouldn’t even be especially confident in it. My point is much more narrow, about how much evidence the outside view should be. There’s also hypothetical ways runaway competition could happen that have nothing to do with AI.
“There’s no guarantee you won’t get outcompeted even if you’ve been dominant in the Darwinian competition so far.” I’m saying something stronger than that, that it’s predictable that you will do better and better for a while, even if you are about to get outcompeted - see below.

Here’s what I AM saying but is not my main point:

“Long-run evolutionary dynamics seem concerning - it seems like there could be a race to the bottom where entities that care a nonzero amount about anything other than gaining power and expanding get outcompeted”: I agree, but this has been said by many people, including Nick Bostrom, Robin Hanson, Allan Dafoe here and here (I’d recommend the latter), Guive Assadi, Dan Hendrycks, Carl Shulman here and here, Andrés Gómez-Emilsson, and even arguably Nick Land.
“It’s possible that humans are in a temporary non-Malthusian Golden Age because of how economically valuable they happen to be right now^[2]” I agree, but this has been touched on by eg Hanson and Bostrom again, Kulveit et al., Korinek and Stiglitz, and Garfinkel^[3].

I hoped to convey an additional thing - here’s the original post’s description of it:

It makes complete sense that this nihilistic optimization process [uncontrollable runaway civilizational growth through Darwinian competition] at first actually benefits some class of agent - because initially, the easiest way to keep growing is to use some class of agent in the world and incentivize it by satisfying its preferen

...