There are many substances that take an enormous amount of time to form. Examples include alcohols, diamonds, and oil. The idea that there might be a process in the human brain that evolved over millions of years and simply cannot be replicated using transistors—or in any other way over the next hundreds of thousands of years—seems more likely than not.

For example, hundreds of years from now, the most advanced AI might have an activation function built into its architecture that it needs to use to remain competitive with other instances, but biology of the human brain can perform it 0.01% more efficiently. Okay, perhaps an activation function isn't the best example, since that could actually be replicated simply by keeping a few neurons alive.

But what do you think is the probability that there are more advanced, complex functions that require consciousness and that, as a result of millions of years of biological evolution, are more energy-efficient than any AI could possibly be over the next few hundred years? And how would that affect p(doom)? What if, in addition to consciousness, such functions also required a certain amount of dopamine and serotonin? And what is the probability of that?

It seems to me that this is one of the most important questions we need to answer if we wonder what p(doom|ASI) is.