The Bio Anchors report by Ajeya Cotra tries to forecast AGI timelines by getting estimates of how much compute might be needed to build or train a model that has general intelligence capabilities. This estimate is based on various biological comparisons, and for the purposes of this question I want to focus on the most conservative estimate from the report, that coming from how much computation was done by evolution in the history of life.

It turns out we can bound this in several different ways by a number on the order of FLOP. One argument could be this: the Landauer limit bound on how much compute can be done on Earth given how much energy the Earth is able to radiate is something like bit erasures/year, and we can cut this down by many orders of magnitude due to various considerations: brains operate far from the Landauer limit at the bit level (though maybe not if you use a larger scale Landauer limit that conditions on irreversible information transmission via electrons, see this post), evolution is only responsible for a small fraction of the Earth's radiation, *et cetera*. Overall I think something on the order of FLOP for total compute done by evolution seems sufficiently big to be safe. The Bio Anchors report uses a different method but arrives at a number that's not very different from this one.

There is, however, a potential problem with this: we can imagine that evolution is an optimization process that's running on many different planets, and only in a select few number of them does evolution end up producing general intelligence. If this is true, producing general intelligence can be a *much harder* problem than this calculation gives it credit for, because anthropic considerations mean we would only be asking the question of how difficult general intelligence is to produce in worlds where general intelligence was actually produced. We should therefore multiply our compute upper bound by the reciprocal of the probability that this amount of compute is actually sufficient to produce general intelligence, and anthropic considerations make it very difficult to put any nontrivial bounds on this probability, threatening to make the entire estimate worthless.

One argument I see on the other side is from Robin Hanson's grabby aliens model. He points out that if evolution happens in a sequence of steps and some of these steps are "hard" in the sense that their expected completion time is much greater than the lifespan of the Earth, and the completion time of each step follows an exponential distribution with unknown mean, then conditioning on all of the steps being successful in time to give rise to intelligent life, these steps will all look like they have the same average length in the history of life on Earth, and moreover this length will also on average equal the time remaining for Earth to become uninhabitable.

As Earth still has many hundreds of millions of years before it becomes uninhabitable due to natural processes, this suggests that anything in evolutionary history that happened in much less time than this could not have been a hard step, and in particular the whole transition from apes to humans happened too quickly in evolutionary time for it to count as a hard step. So if we think the "juice" of general intelligence is in the last few million years, this argument could suggest anthropic considerations are not important after all, but I remain unsure about this.

What do you think about this problem? I'm curious to hear people's thoughts.

I agree in general with this point, but in this context we have a problem even if the scaling is something extremely favorable like ~ log log N instead of ~ N, at least under the assumption that the universe is infinite.

The problem is really that we don't know N and it could be arbitrarily large in an infinite universe, so while I agree linear scaling is too pessimistic of an assumption and the right scaling is probably more like a power law with smaller exponent, I don't see how any scaling that's not bounded from above in N is going to get around the anthropic problems.

I'm just generally confused by anthropics and I made this post to get the opinions of other people on whether this is actually a problem or not.