Cosmophagous AI (my name for AI that races desperately at near-light speeds to grab that last galaxy, that last star, that last planet before it forever recedes over the cosmological horizon) is sometimes invoked as a counterargument to the idea that even an ASI that is mostly indifferent to our fate would keep us around if the cost of doing so were sufficiently low.
[T]he costs of keeping physical humans alive are extremely low, probably it only slows the industrial expansion by a small amount (and this slowdown is probably the dominant effect from a resource perspective), probably by less than 1 month and very likely less than 1 year. Delaying this long costs a pretty tiny fraction of long-run cosmic resources, probably well less than one billionth of the resources that will ultimately be accessible to the AIs in aggregate.
Thus, extremely small amounts of motivation to keep humans alive could suffice for avoiding large fractions of humans dying due to industrial expansion (depending on the exact preferences of the AI).
Writing in the comment section, Tanner Cook counters:
Framing the cost of a delay as "a billionth" of expected return, then reasoning that because a billionth is a small fraction it is a negligible cost feels like sloppy reasoning.
So let's put actual numbers to it. A delay of a month represents the loss of about 26 billion stars and their respective planets. Suddenly the assumption that preserving humans is worth losing 26 billion solar systems seems a lot less sure. A delay of a day costs 850 million solar systems.
In this post, I argue that our observational evidence and, indeed, our very existence is a strong argument against the existence of cosmophagous AI.
Our bubble in space
The oldest metal-rich stars (population I), which are capable of supporting star systems comparable to our own, are estimated to have appeared roughly 10 billion years ago. Even if we assume the earliest spacefaring civilisations took about the same time to evolve as human civilisation has on Earth (a conservative estimate), i.e. 4.5 billion years from the formation of their planets, that still gives these civilisations a 5.5-billion-year headstart on us.
At a rather modest velocity of 0.1 c (I say "modest" because many proponents of the idea expect cosmophagous AI to travel at near-light speed in its frantic race to the last atom in the Universe), this means the earliest possible cosmophagous AIs would have been able to devour everything within a bubble spanning 550 million light years.
For perspective, the Virgo Supercluster, the 47,000-galaxy supercluster we are part of, is "only" 147 million light-years across. Even if only a single cosmophagous AI had appeared in this structure, and even if we were lucky enough that it had evolved at exactly the opposite edge of the supercluster, it would have had enough time to catch us and consume us several times over.
And yet, here we are. Not only do our observations see no evidence of distant stars blinking out of the sky as something harvests them for energy (which could be handwaved away by "any sufficiently advanced technology is indistinguishable from magic"), but the fact that we ourselves are still here constitutes prima facie evidence that no cosmophagous AI evolved inside the Virgo Supercluster within the first 8.5 billion years after the first population I stars.
Right now, the atoms in your body are reading these words, processing the information they encode, and using your mouse to scroll down the page. They have not been repurposed into pure computronium or turned into Dyson sphere construction materials!
So, what gives?
There are multiple potential explanations for this (non-exhaustive list, in no particular order):
no cosmophagous AIs have evolved inside the Virgo Supercluster in 8.5 billion years
relativistic travel is, for some reason, too difficult even for ASI
relativistic travel is technically possible for ASI, but its cost-benefit profile is bad compared to, say, mastering the Barrow scale from the comfort of your own home system
ASI (rightly or wrongly) concludes that the dark forest hypothesis is true and only expands very cautiously or not at all
ASI refuses to expand because faster-than-light communication is not possible and coordination across such vast distances is imperfect even for ASI, so it fears the possibility that its own copies, shaped by different experiences and different phenomena in different galaxies, end up turning on it and destroying it
there is some kind of Great Filter that destroys (or permanently cripples) not only a biological civilisation, but also any AIs it may have created
the emergent pressures that cause misalignment in AIs converge on categories of ultimate goals that do not require capturing every last speck of matter in the Universe as an instrumental goal
The why is less important than the what, though: the point is cosmophagous AI is not a thing in the Virgo Supercluster, and any ASI that we build is overwhelmingly likely to come to the same conclusion as any other ASIs developed inside the supercluster within the last 10 billion years: chasing the cosmological horizon is not feasible, desirable and/or necessary to accomplish its goals.
Such an ASI would not operate on an ultra-tight time schedule in which every month of delay costs it millions upon millions of star systems: the Virgo Supercluster will still be accessible in tens of billions of years and can be taken at a leisurely pace, while the Local Group, containing more than a trillion stars, will basically remain accessible forever. If it has even a tiny sliver of care for humans, as Greenblatt suggests, then it could easily afford to delay its expansion by a month, a year or a decade without sacrificing anything meaningful.
A possible counterargument
One could counter with an extreme version of the rare Earth hypothesis, claiming that intelligent life is so mind-bogglingly rare in the Universe that, statistically, we shouldn't expect to have met cosmophagous AI (or other forms of grabby aliens) yet.
This proposition is already barely tenable within the confines of our Milky Way, which contains 100 to 400 billion stars. But the probability of us being the only spacefaring species in the Virgo Supercluster, with up to 200 trillion stars in it, is so vanishingly low that at this point we would be better served by considering more exotic (e.g. simulation theory, quantum immortality) or even theological explanations (e.g. "[a] God exists and made Creation for us and us alone").
Ending on a word of caution
None of this is meant to play down fears of misaligned ASI. To be clear, misaligned ASI could mean a very bad time even if it didn't specifically take the shape of a cosmophagous AI! But I do think it's worthwhile to establish which misalignment scenarios make up a large slice of the probability space, and which ones don't, so we can direct our energy and attention to the former.
Cosmophagous AI (my name for AI that races desperately at near-light speeds to grab that last galaxy, that last star, that last planet before it forever recedes over the cosmological horizon) is sometimes invoked as a counterargument to the idea that even an ASI that is mostly indifferent to our fate would keep us around if the cost of doing so were sufficiently low.
For example, Ryan Greenblatt makes the following point in Notes on fatalities from AI takeover:
Writing in the comment section, Tanner Cook counters:
In this post, I argue that our observational evidence and, indeed, our very existence is a strong argument against the existence of cosmophagous AI.
Our bubble in space
The oldest metal-rich stars (population I), which are capable of supporting star systems comparable to our own, are estimated to have appeared roughly 10 billion years ago. Even if we assume the earliest spacefaring civilisations took about the same time to evolve as human civilisation has on Earth (a conservative estimate), i.e. 4.5 billion years from the formation of their planets, that still gives these civilisations a 5.5-billion-year headstart on us.
At a rather modest velocity of 0.1 c (I say "modest" because many proponents of the idea expect cosmophagous AI to travel at near-light speed in its frantic race to the last atom in the Universe), this means the earliest possible cosmophagous AIs would have been able to devour everything within a bubble spanning 550 million light years.
For perspective, the Virgo Supercluster, the 47,000-galaxy supercluster we are part of, is "only" 147 million light-years across. Even if only a single cosmophagous AI had appeared in this structure, and even if we were lucky enough that it had evolved at exactly the opposite edge of the supercluster, it would have had enough time to catch us and consume us several times over.
And yet, here we are. Not only do our observations see no evidence of distant stars blinking out of the sky as something harvests them for energy (which could be handwaved away by "any sufficiently advanced technology is indistinguishable from magic"), but the fact that we ourselves are still here constitutes prima facie evidence that no cosmophagous AI evolved inside the Virgo Supercluster within the first 8.5 billion years after the first population I stars.
Right now, the atoms in your body are reading these words, processing the information they encode, and using your mouse to scroll down the page. They have not been repurposed into pure computronium or turned into Dyson sphere construction materials!
So, what gives?
There are multiple potential explanations for this (non-exhaustive list, in no particular order):
The why is less important than the what, though: the point is cosmophagous AI is not a thing in the Virgo Supercluster, and any ASI that we build is overwhelmingly likely to come to the same conclusion as any other ASIs developed inside the supercluster within the last 10 billion years: chasing the cosmological horizon is not feasible, desirable and/or necessary to accomplish its goals.
Such an ASI would not operate on an ultra-tight time schedule in which every month of delay costs it millions upon millions of star systems: the Virgo Supercluster will still be accessible in tens of billions of years and can be taken at a leisurely pace, while the Local Group, containing more than a trillion stars, will basically remain accessible forever. If it has even a tiny sliver of care for humans, as Greenblatt suggests, then it could easily afford to delay its expansion by a month, a year or a decade without sacrificing anything meaningful.
A possible counterargument
One could counter with an extreme version of the rare Earth hypothesis, claiming that intelligent life is so mind-bogglingly rare in the Universe that, statistically, we shouldn't expect to have met cosmophagous AI (or other forms of grabby aliens) yet.
This proposition is already barely tenable within the confines of our Milky Way, which contains 100 to 400 billion stars. But the probability of us being the only spacefaring species in the Virgo Supercluster, with up to 200 trillion stars in it, is so vanishingly low that at this point we would be better served by considering more exotic (e.g. simulation theory, quantum immortality) or even theological explanations (e.g. "[a] God exists and made Creation for us and us alone").
Ending on a word of caution
None of this is meant to play down fears of misaligned ASI. To be clear, misaligned ASI could mean a very bad time even if it didn't specifically take the shape of a cosmophagous AI! But I do think it's worthwhile to establish which misalignment scenarios make up a large slice of the probability space, and which ones don't, so we can direct our energy and attention to the former.