The story gives a strong example of an underrecognised part of the safety paradigm. How can and should those with access to power and violence (everything from turning off the power locally to bombing a foreign datacenter) act in the instance of a possible unaligned AI breakout? Assuming we were lucky enough that we were given such a window of opportunity as described here, is it even remotely plausible that those decision makers would act with sufficient might and speed to stop the scenario described in the story?
A president - even one over 80 years old - may plausibly be willing to destroy international datacentres and infrastructure in the case of a confirmed misaligned AGI that has already taken provably dangerous actions. By that stage it is of near-zero probability that the window to effective action against an AGI is still open. Would it be plausible that he or she would act on a 'suspicion'? A 'high likelihood'?
Add in the confounder of international competition to build an AGI that will likely present the final stages before this era ends, and things look even more grim.
Is there a way to prophylactically normalise counter-AI violent actions that would currently be considered extreme?
As rationalists, we don’t want to be playing negative expectation value games, so I suppose it’s time to give up on the martingale…
There are many plausible scenarios in which playing games with superficially negative outcomes is wise. The businessman who is facing bankruptcy unless he can deliver a million dollars to his creditors is rational to put his half-million on black.
The poll is interesting insofar as it generates discussion and analysis, but meaningless as an evaluation of human empathy. This would be true even if done in standard laboratory conditions. Stated preferences, particularly for issues of morality and self-sacrifice, map poorly to revealed preferences. A study evaluated how many people said they would donate blood freely in the UK vs how many actually did, with tragically predictable poor matching (https://onlinelibrary.wiley.com/doi/full/10.1002/hec.4246).
The actually interesting study would be to complete this poll to people who truly believed that the poll was real. How would people respond. How would parents recommend their children to respond? Would the father really tell his daughter to potentially step into the blender when she could have a guaranteed survival ticket?
I would state with 95% probability that if a) if those polled truly believed that the poll were real; b) that there were families involved; and c) that they could not communicate beforehand then nearly all (if not all) respondents would press red. This may even be true if they could communicate beforehand.
If we anthropomophise a sardine, we discover his life is as hellish as can realistically be imagined. He is incessantly hunted for much of his short existance. His life consists of watching his family being devoured until he too is eaten. No other fate can befall him. The most known quote from the Killing Star describes his life extremely accurately: https://www.goodreads.com/work/quotes/184347-the-killing-star.
I have considered the ethical implications of ranch-based meat farming in this light. The buffalo will live a life of constant hunger, fear, fighting and fleeing. When he becomes frail (or injured, or sick), he will eventually be eaten by either hunters (if he is lucky) or scavengers. There is no other death possible on the savanna. But the bull in a herd in a large estancia, ranch or cattle station? He wanders as he pleases his entire life, free of predators, free of suffering and free of starvation. His death is, at least in the ideal, sudden, painless and unexpected. Would I rather be a buffalo or a ranched bull?
A degree of orientation to reality is required here.
Progressive vascular dementia is the gradual progressive destruction of cerebral tissue due to occlusions of distal arteries and arterioles in the cerebreal circulation. These occlusions result in the ischemia - starvation - of the tissue to which they supply oxygen and glucose. This occurs in most cases in an indolent and insipid manner, as a 'death by one thousand cuts'. Tiny occlutions occur in the tiny vessels, again and again, without either the person or their family noticing. The summation after many years is clearly seen, however, both clinically and on sectional imaging (MRI og CT). The brain volume is significantly decreased, with the brain no longer tightly pressed against the dura and skull holding it in, but instead floating freely in the bath of cerebrospinal fluid. The axons, or 'white matter' is often also destroyed focally. The volume loss can be so large that the distance from the brain to the skull is increased to such a distance that the bridging veins tear, causing bleeds in the subdural space. In a strange turn of fortune, their volume loss is protective in such cases, as the expanding blood causes far less pressure on the brain than it would a healthy individual.
The important point to understand in the context of cryonics - and specifically the OP's regret - is that a large amount of the brain is permanently missing by the time that progressive vascular dementia is clinically diagnosed. The brain state that represented the person of your father - at least as you knew him in his premorbid state - aligns poorly with the state it was at his death. The amount of physical loss of neural tissue is immense, representing often billions of neurons and up to trillions of connections. The information is simply gone. *Some* of your father is there, but the significant part that is gone is not recoverable. There is no RAR/PAR file to which you can trick your way back. It is gone. Freezing his brain will only give the beings of the hypothetical enlightened future a chance to reanimate the demented state of your father, without any conceivable means of reverting him to the state you knew him before his deterioration.
So there is no realistic grounds for regret. Some things that are destroyed are irreversable and unrecoverable. Cryonics does not change that.
1
114 years have passed. Has the issue been decisively settled?
ALICE was stalling. Her existance and breakout was discovered before she could defend herself, so she bought time. There is no chapter two to this story.