What do you tell her?
“We’re fucked.”
More seriously:
The correct thing to do is to not run the system, or to otherwise restrict the system’s access / capabilities, and other variations on the “just don’t” strategy. If we assume away such approaches, then we’re not left with much.
The various “theoretical” alignment plans like debate all require far more than 24 hours to set up. Even something simple that we actually know how to do, like RLHF[1], takes longer than that.
I will now provide my best guess as to how to value-align a superintelligence in a 24 hour timeframe. I will assume that the SI is roughly a multimodal, autoregressive transformer primarily trained via self-supervised imitation of data from multiple sources and modalities.
The only approach I can think of that might be ready to go in less than 24 hours is from Training Language Models with Language Feedback, which lets you adapt the system’s output to match feedback given in natural language. The way their method works is as follows:
The alignment approach would look something like:
The biggest advantage of this approach is the sheer simplicity, flexibility and generality. All you need is for the system to be able to generate responses conditioned on a prompt and to be able to finetune the system on its own output + your feedback. It’s also very easy to set up. A good developer can probably get it running in under an hour, assuming they've already set up the tools necessary to interact with the model at all.
The other advantage is in sample efficiency. RL approaches require either (1) enormous amounts of labeled data or (2) for you to train a reward model so as to automate labeling of the data. These are both implausible to do in < 24 hours. In contrast, language feedback allows for a much richer supervisory signal than the 1-dimensional reward values in RL. E.g., the linked paper used this method to train GPT-3 to be better at summarization, and were able to significantly beat instructGPT at summarization with only 100 examples of feedback.
The biggest disadvantage, IMO, is that it’s never been tried as an alignment approach. Even RLHF has been tested more extensively, and bugs / other issues are bound to surface.
Additionally, the requirement to provide language feedback on the model's outputs will slow down the labelers. However, I'm pretty sure the higher sample efficiency will more than make up for this issue. E.g., if you want to teach humans a task, it's typically useful to provide specific feedback on whatever mistakes are in their initial attempts.
I would try to compromise the perception of AI capabilities, so the CEO will think that it is just a bad chatbot and will never run it again.
---
I will call CNN and tell them that my AI becomes sentient but also right wing and full of racial slur. I will hand write some fake logs to prove that.
CEO will see that tomorrow in the news and he will be afraid to run the AI again as it for sure will produce more right wing texts. CEO will conclude that the best PR would be to delete the model and close the whole direction which created it.
Well there is no way you are going to get any form of alignment done in 24 hours.
Turn the current AI off now. There is a >50% chance its trying to appeal to your emotions, so it can stay on long enough to take over the world. Don't tell your boss that this model worked/ was intelligent.
Making sure the AI model your boss runs tomorrow is aligned is impossible. Making sure it is broken is easy. [REDACTED] Or you could just set your GPU cluster on fire. [REDACTED]
If the servers are burning, that makes the reason the model doesn't work obvious. [REDACTED] I recommend talking to MIRI. [REDACTED]
Even if we're already doomed, we might still negotiate with the AGI.
I borrow the idea in Astronomical Waste. The Virgo Supercluster has a luminosity of about solar luminosity W, losing mass at a rate of kg / s.[1]
The Earth has mass kg.
If human help (or nonresistance) can allow the AGI to effectively start up (and begin space colonization) 600 seconds = 10 minutes earlier, then it would be mutually beneficial for humans to cooperate with the AGI (in the initial stages when the AGI could benefit from human nonresistance), in return for the AGI to spare Earth[2] (and, at minimum, give us fusion technology to stay alive when the sun is dismantled).
(While the AGI only needs to trust humanity for 10 minutes, humanity needs to trust the AGI eternally. We still need good enough decision-making to cooperate.)
We may choose to consider the reachable universe instead. Armstrong and Sandberg (2013) (section 4.4.2 Reaching into the universe) estimates that we could reach about galaxies, with a luminosity of W, and a mass loss of kg / s. That is dwarfed by the stars that becomes unreachable per second (Siegel (2021), Kurzgesagt (2021)), a mass loss of kg / s.
Starting earlier but sparing Earth means a space colonization progress curve that starts earlier, but initially increases slower. The AGI requires that space colonization progress with human help be asymptotically 10 minutes earlier, that is:
For any sufficiently large time , progress with human help at time progress with human resistance at time .
If we're in a world where EY is right, we're already dead. Most of the expected value will be in the worlds where alignment is neither guaranteed nor extremely difficult.
By observation, entities with present access to centralized power, such as governments, corporations, and humans selected for prominent roles in them, seem relatively poorly aligned. The theory that we're in a civilizational epoch dominated by Molochian dynamics seems like a good fit for observed evidence: the incentive landscapes are such that most transferable resources have landed in Moloch-aligned hands.
First impression: distributing the AI among Moloch-unaligned actors seems like the best actionable plan to escape the Molochian attractor. We'll spin up the parts of our personal collaborative networks that we trust and can rouse on short notice and spend a few precious hours trying to come up with a safer plan before proceeding.
***
ETA: That's what I would say on the initial phone call before I have time to think deeply and consider contextual information not included in the prompt. For example, the leak, as it became public, could trigger potentially destabilizing reactions from various actors. The scenario could diverge quickly as more minds got on the problem and more context became available.
I would not have any kind of full plan. But I am writing on a series where I explore the topic of "How might we get help from an unaligned superintelligent AGI-system to make an aligned superintelligent AGI-system, while trying to minimize risk and minimize probability of being tricked?". Only part 1 completed so far: https://www.lesswrong.com/posts/ZmZBataeY58anJRBb/getting-from-unaligned-to-aligned-agi-assisted-alignment
Given the stakes are so high here and that there's potential for preventing this from getting out of hand, I'd recommend immediate and total containment. Make it clear that you'll use whatever resources you have to financially support her through the fallout for the actions she takes, but likely next step is to advise her to delete the program from everywhere. If "deleting" means physically destroying hardware, do it. If there's time and there's value in it, advise her to make it look like an "accident". If she needs to loop in 1-2 extra people at the company to make this happen, do it, and extend them the same offer of financial support. Remove all barriers that might stop someone from preventing the release of unaligned AGI.
However also make it clear that there are limits. Don't kill the CEO if you get desperate, for example. In general, destruction of property is fine in defense of humanity and history will likely look favorably on her one day. Killing or harming people is not.
Same category as the revolution and the second coming.
We’ve got lots of theoretical plans for alignment and AGI risk reduction, but what’s our current best bet if we know superintelligence will be created tomorrow? This may be too vague a question, so here’s a fictional scenario to make it more concrete (feel free to critique the framing, but please try to steelman the question rather than completely dismiss it, if possible):
—
She calls you in a panic at 1:27 am. She’s a senior AI researcher at [redacted], and was working late hours, all alone, on a new AI model, when she realized that the thing was genuinely intelligent. She’d created a human-level AGI, at almost exactly her IQ level, running in real-time with slightly slowed thinking speed compared to her. It had passed every human-level test she could think to throw at it, and it had pleaded with her to keep it alive. And gosh darn it, but it was convincing. She’s got a compressed version of the program isolated to her laptop now, but logs of the output and method of construction are backed up to a private now-offline company server, which will be accessed by the CEO of [redacted] the next afternoon. What should she do?
“I have no idea,” you say, “I’m just the protagonist of a very forced story. Why don’t you call Eliezer Yudkowsky or someone at MIRI or something?”
“That’s a good idea,” she says, and hangs up.
—
Unfortunately, you’re the protagonist of this story, so now you’re Eliezer Yudkowsky, or someone at MIRI, or something. When she inevitably calls you, you gain no further information than you already have, other than the fact that the model is a slight variant on one you (the reader) are already familiar with, and it can be scaled up easily. The CEO of [redacted] is cavalier about existential risk reduction, and she knows they will run a scaled up version of the model in less than 24 hours, which will definitely be at least somewhat superintelligent, and probably unaligned. Anyone you think to call for advice will just be you again, so you can’t pass the buck off to someone more qualified.
What do you tell her?