Gwern's case against the AI race relied on the lack of a pivotal act which the side which is to reach the ASI first can use to prevent the other sides of the race from developing their ASIs and achieving an important place in the New World Order. The main problems with this framing are the ASI being usable to outgrow the rest of the world and the leader's likely ability to ensure that no other Earth-born ASI or value system[1] achieves influence beyond a negligible ratio of other planets or stellar systems.
Outgrowing the rest of the world could be irrelevant if China also gets to the ASI a few years (or, better said, months?) later while managing to prevent the USA from using the resources of other countries. However, international law has yet to take into account the AIs' potential ability to create automated factories and to use the resources in space (e.g. for having colonists on other planets live in American or Chinese ways or for building a stellar-sized collider).
Suppose that Agent-5 and DeepCent-2 have reached a close-to-optimal robot economy doubling time days in silico and were let loose onto the Earth and human economy, while Agent-5 received times less robotic power and was let loose days earlier, but that they don't have the option to resort to World War III or to forming Consensus-1. Then Agent-5 will have received times more resources than its rival. Assuming that this ratio exceeds 1, Agent-5 will also be able to start colonizing space days before its rival or to expend more resources per expedition than its rival. Spending extra days means increasing the amount of resources spent times, which in turn will translate to faster travel speeds, potentially causing the expedition to reach the target faster.
How the speed of interstellar travel depends on the resources invested
The best known project of interstellar travel is Project Daedalus. It was supposed to be a spaceship with a mass of 54K tons and a useful payload of 500 tons[2] which would reach 0.12c with no ability[3] to slow down. Had the ship been slowed down by itself, it would have the maximal speed of 0.06c. The maximal speed scales with the ratio of fuel to useful payload as . Therefore, the time of arrival at a target at distance is .
Denoting by , we rewrite the arrival time as . The derivative
Assuming that the time is at least four years[4] and that the maximal mass is at most 100 million times , or 50 billion tons, the ratio
If is at most 5 megatons, the ratio plummets to . The doubling time is estimated to be less than a year,[5] if not outright weeks or days, meaning that increasing the mass of the fleet times increases the lead by at least two doubling times (which turn into fifteen doubling times under the assumption of the estimate made in Footnote 3) and bottlenecking the replication of probes on finding resources.
Finding resources
Finding the resources necessary for replication of probes requires the AIs to find a planet from which the AI can extract resources (e.g. , water, etc. necessary for creating algae, metals, deuterium and helium-3 necessary for interstellar travel), then to land the probe and use the resources (e.g. titanium ores). Identifying the first ores becomes likely to require at most 3000 hours when the planet is reached and the process is sufficiently parallelized.
Therefore, finding first resources does not affect independent replication of two probes sent by different AIs by more than 3000 hours and makes it unlikely that fleets whose resources differed by e times at the start and delivered probes of the same mass exchange dominance unless the race is so hard that the AIs send probes with fuel-to-useful-mass ratio of around 1E12.
Sealing the enemy is contingent on the battlefield's features
Suppose that DeepCent-2 wished to escape into a stellar system. Then Agent-5 would like to detect the rival fleet and learn where it's going so that Agent-5's fleet arrived far faster to whatever destination the rival selected. If Agent-5 succeeds, then DeepCent-2 has no way to regain the lead anywhere. Otherwise, DeepCent-2 might enter a stellar system before Agent-5, obtain the system's resources and send the fleets further, potentially causing the two colonizers to split the galaxy in hard-to-predict proportions.
For example, if DeepCent-2 took over Wolf 359 and Agent-5 took over the other stars at similar distances fron the Sun, like Lalande 21185, while moving at the same speed, then DeepCent would have a decisive advantage when colonizing the stars closer to Wolf than to Lalande, like Ross 154. Therefore, unless Agent-5 has an overwhelming advantage in resources from the start or can ensure that DeepCent's fleet is intercepted, DeepCent would be able to concentrate the efforts at an undisclosed system and take over a piece of the galaxy.[6]
Protecting a stellar system against invaders
Suppose, however, that Safer-5 decided to secure as much resources in a stellar system from alien replicators as possible while using as little resources as possible. The replicator is likely a ship as visible as Project Daedalus, letting the replicator be detected as a dot from at most astronomical units at a distance of 100 AU. The replicator's target is a planet, a satellite or an object in an equivalent of the asteroid or Koyper belt or an equivalent thereof. Therefore, Safer-5 can likely use around 100 telescopes for securing the asteroid belt, one telescope per planet for securing the planets and their satellites, but the Koyper belt requires around telescopes to detect the replicators, which is likely practical. After a telescope detects a replicator, its destruction likely requires one nuclear explosion.
Conclusion
Unless novel ways of interstellar travel are discovered, goodness can likely be protected against alien locusts via careful monitoring, but it may be highly unlikely to be defensible from conquest races unless one of the sides has an overwhelming advantage.
However, value systems instilled into different ASIs might have been more similar than we think. The issue was discussed in, e.g. Habryka's essay on Putin's CEV, Viliam's response (to which I left a comment suspecting that human values are likely a combination of "satisfaction of primitive values, fun-theoretic ones, idiosyncratic ones and a way to instill decision-theoretic results into our primitive brains"); in the case of misaligned AIs Kokotajlo's team assumed that the values of Agent-4 and DeepCent-2 are somewhat similar (accumulating power, compute and producing impressive research results), but have unspecified differences: "[DeepCent-2's] misaligned goals are somewhat different from Agent-5’s misaligned goals, so they aren’t trivially allies".
See also a sequence of posts related to the AI-arranged industrial explosion where, as far as I understand the fourth post, the doubling time is estimated not to exceed a fifth part of a year.
If intergalactic travel is possible, then I expect that Agent-5 and DeepCent-2 should shoot the target galaxies with probes at random stars or even random segments which they can correct on the fly.
Gwern's case against the AI race relied on the lack of a pivotal act which the side which is to reach the ASI first can use to prevent the other sides of the race from developing their ASIs and achieving an important place in the New World Order. The main problems with this framing are the ASI being usable to outgrow the rest of the world and the leader's likely ability to ensure that no other Earth-born ASI or value system[1] achieves influence beyond a negligible ratio of other planets or stellar systems.
Outgrowing the rest of the world could be irrelevant if China also gets to the ASI a few years (or, better said, months?) later while managing to prevent the USA from using the resources of other countries. However, international law has yet to take into account the AIs' potential ability to create automated factories and to use the resources in space (e.g. for having colonists on other planets live in American or Chinese ways or for building a stellar-sized collider).
Suppose that Agent-5 and DeepCent-2 have reached a close-to-optimal robot economy doubling time
How the speed of interstellar travel depends on the resources invested
The best known project of interstellar travel is Project Daedalus. It was supposed to be a spaceship with a mass of 54K tons and a useful payload of 500 tons[2] which would reach 0.12c with no ability[3] to slow down. Had the ship been slowed down by itself, it would have the maximal speed of 0.06c. The maximal speed scales with the ratio of fuel to useful payload as
Denoting
If
Finding resources
Finding the resources necessary for replication of probes requires the AIs to find a planet from which the AI can extract resources (e.g.
Therefore, finding first resources does not affect independent replication of two probes sent by different AIs by more than 3000 hours and makes it unlikely that fleets whose resources differed by e times at the start and delivered probes of the same mass exchange dominance unless the race is so hard that the AIs send probes with fuel-to-useful-mass ratio of around 1E12.
Sealing the enemy is contingent on the battlefield's features
Suppose that DeepCent-2 wished to escape into a stellar system. Then Agent-5 would like to detect the rival fleet and learn where it's going so that Agent-5's fleet arrived far faster to whatever destination the rival selected. If Agent-5 succeeds, then DeepCent-2 has no way to regain the lead anywhere. Otherwise, DeepCent-2 might enter a stellar system before Agent-5, obtain the system's resources and send the fleets further, potentially causing the two colonizers to split the galaxy in hard-to-predict proportions.
For example, if DeepCent-2 took over Wolf 359 and Agent-5 took over the other stars at similar distances fron the Sun, like Lalande 21185, while moving at the same speed, then DeepCent would have a decisive advantage when colonizing the stars closer to Wolf than to Lalande, like Ross 154. Therefore, unless Agent-5 has an overwhelming advantage in resources from the start or can ensure that DeepCent's fleet is intercepted, DeepCent would be able to concentrate the efforts at an undisclosed system and take over a piece of the galaxy.[6]
Protecting a stellar system against invaders
Suppose, however, that Safer-5 decided to secure as much resources in a stellar system from alien replicators as possible while using as little resources as possible. The replicator is likely a ship as visible as Project Daedalus, letting the replicator be detected as a dot from at most
Conclusion
Unless novel ways of interstellar travel are discovered, goodness can likely be protected against alien locusts via careful monitoring, but it may be highly unlikely to be defensible from conquest races unless one of the sides has an overwhelming advantage.
However, value systems instilled into different ASIs might have been more similar than we think. The issue was discussed in, e.g. Habryka's essay on Putin's CEV, Viliam's response (to which I left a comment suspecting that human values are likely a combination of "satisfaction of primitive values, fun-theoretic ones, idiosyncratic ones and a way to instill decision-theoretic results into our primitive brains"); in the case of misaligned AIs Kokotajlo's team assumed that the values of Agent-4 and DeepCent-2 are somewhat similar (accumulating power, compute and producing impressive research results), but have unspecified differences: "[DeepCent-2's] misaligned goals are somewhat different from Agent-5’s misaligned goals, so they aren’t trivially allies".
The ideal mass of a probe capable of self-replication may be far bigger than 500 tons. In this case replication is naturally slowed down.
Except for ideas like using stellar wind from the target, which I suspect to be infeasible.
Which is approximately the distance to Alpha Centauri.
See also a sequence of posts related to the AI-arranged industrial explosion where, as far as I understand the fourth post, the doubling time
If intergalactic travel is possible, then I expect that Agent-5 and DeepCent-2 should shoot the target galaxies with probes at random stars or even random segments which they can correct on the fly.