I played the AI box game as the Gatekeeper — and lost
31Zach Stein-Perlman
2Nathan Helm-Burger
4Zach Stein-Perlman
9Matt Goldenberg
4datawitch
1[anonymous]
23Hide
42Bojadła
2lsusr
2the gears to ascension
1[comment deleted]
4k64
3interstice
1Morpheus
0the gears to ascension
7Carl Feynman
2the gears to ascension
15Radford Neal
16Dagon
14Nathan Helm-Burger
17the gears to ascension
13Viliam
6datawitch
32gjm
30Daniel Kokotajlo
7[anonymous]
3Measure
2[anonymous]
13NeroWolfe
9datawitch
1PhilosophicalSoul
3Malentropic Gizmo
14GoteNoSente
4tgb
13the gears to ascension
11Linch
5tslarm
2datawitch
1red75prime
1datawitch
5Measure
2datawitch
3PhilosophicalSoul
2Jiro
2datawitch
2Jiro
2datawitch
2[comment deleted]
2tslarm
1Double
2CronoDAS
1tslarm
4datawitch
1Gesild Muka
1tslarm
4datawitch
New Comment
Unfortunately, I can't talk about the game itself, as that's forbidden by the rules.
You two can just change the rules... I'm confused by this rule.
The implication is that they approved of this rule and agreed to conduct the social experiment with that rule as part of the contract. Is that not your understanding?
I agree. But I claim saying "I can't talk about the game itself, as that's forbidden by the rules" is like saying "I won't talk about the game itself because I decided not to" -- the underlying reason is unclear.
The original reasoning that Eliezer gave if I remember correctly was that it's better to make people realize there are unknown unknowns, rather than taking one specific strategy and saying "oh, I know how I would have stopped that particular strategy"
3
3Another reason for such a rule could be to allow the use of basilisk-like threats and other infohazards without worrying about them convincing others beyond the gatekeeper.
That said, @datawitch @ra I'm interested in reading the logs if you'd allow.
1I cannot imagine losing this game as the gatekeeper either, honestly.
Does anyone want to play against me? I’ll bet you $50 USD.
I agree and I am putting my money where my mouth is.
I will play this game under the rules linked in the OP with me as the gatekeeper and anyone as the AI. I will bet at odds 10:1 (favorable to you) that I will not let the AI out. The minimum bet is my 10 k USD against your 1 k USD and the maximum bet my 100 k USD against your 10 k USD. We will use a mutually accepted LW community member as referee.
If you believe you have at least a 10% chance of making me let the AI out, you should take this bet. I predict no one will take me up on it.
I speculate that the reason that gatekeepers let the AI out is that they do not take the experiment seriously enough. They don't really care about not letting the AI out. Maybe they want the outcome to be that the AI has been let out so that people take AI safety more seriously. I'm not saying an actual superintelligence couldn't do it, but no currently available intelligence can (with me as the gatekeeper).
5
2I will bet at odds 10:1 (favorable to you) that I will not let the AI out
I too am confident enough as gatekeeper that I'm willing to offer similar odds. My minimum and maximum bets are my $10,000 USD vs your $1,000 USD.
I don't care to compete because I expect to lose, but strategies that might work against this seem like they'd look like investment pitches. that's typically what gets people to move that much money, after all.
Ok, how are people losing as gatekeepers? I can't imagine losing as a gatekeeper, so I have to think that they must not be trying or they must be doing it for a publicity benefit. I'll give anyone $100 if they can convince me to let them out of the box. Judging by the comments here though, I'm guessing no one will step up and that one of the alternative explanations (publicity, publication bias, etc.) is responsible for the posted results.
Note: if anyone actually is considering playing against me, I will let you interview me beforehand and will honestly answer any questions, even about weakspots or vulnerabilities. I'm also open to negotiating any rules or wager amounts. The only rules I'm attached to are: No real world consequences (including social/moral) and the gatekeeper is not required to let the AI out, even if they believe their character might have in the situation. The only thing I ask is that you be confident you can win or have won before because I want someone to prove me wrong. If you're not confident and have no track record, I am still willing to play, but I may ask you to put up some ante on your side to make sure you take it seriously.
I also don't think I would lose as the gatekeeper(against a human), and would be willing to make a similar bet if anyone's interested.
I'd also bet $50 as a gatekeeper. I won this game as a gatekeeper before and now need someone to put my ego in place. I'd prefer to play against someone who won as the AI before.
This post prompted me to wonder to which degree there might be publication bias going on in that people don't report when they "predictably" won as the gatekeeper (as I did).
3I can think of lots of ways to manipulate someone but I can't think of any that would win this game with the constraint that I have to sound like GPT4
That's not a constraint. The game is intended to provide evidence as to the containment of a future superhuman intelligence. GPT-4 is a present-day subhuman intelligence, and couldn't do any harm if it got out.
fair enough. for the record, since people started proposing bets, I quickly realized I don't really expect to be able to manipulate anyone even that well, so the point is moot in any case.
I don't get it.
Apparently, the idea is that this sort of game tells us something useful about AI safety.
But I don't get it.
You obviously knew that you were not unleashing a probably-malign superintelligence on the the world by letting Ra out. So how does your letting Ra out in this game say anything about how you would behave if you did think that (at least initially)?
So I don't get it.
And if this does say something useful about AI safety, why is it against the rules to tell us how Ra won?
I don't get it.
I think the debate about AI defense strategies has moved well past the idea of "well, just keep it locked up where it can't reach it's own power switch or make any choice that impact itself".
I agree that this was never a particularly compelling demonstration, especially without revealing the "one weird trick" that the AI players use to get out. But I was also never of the opinion that it'd work anyway.
It's still mildly interesting when humans fail to predict even other humans' ability to socially-engineer their behavior. I don't think it says much about AI safety, but it does say something about human vulnerabilities.
I disagree. I think that there is an important point to be made here about AI safety. A lot of people have made the argument that 'an agent which can only work by communicating via text on a computer screen, sent to a human who has been pre-warned to not let it out' can never result in a reasonable thoughtful intelligent human choosing to let it out. I think that the fact that this experiment has been run a few times, and sometimes results in the guardian loosing presents at least some evidence that this claim of persuasion-immunity is false. I think this claim still matters for AI safety, even in the current world. Yes, the frontier labs have been their models access the internet, and communicate to the wider world. But they have not been doing so without safety screening. This safety screening involves, at least in part, a human reading text produced by the model and deciding 'it is ok to approve this model for contact with the world'. Currently, I think that the frontier labs contain a lot of employees who would be more in agreement with the description of themselves as not vulnerable to being persuaded to incorrectly approve a model for release. I think this is overconfidence on their part. Humans are vulnerable to being tricked and conned. I'm not saying that nobody can ever avoid being persuaded, just that we can't assume the robustness of this safety measure and should devise better safety measures.
ok sure, but this game is still terrible and is evidence against clear thinking about the problem at hand on the part of anyone who plays it as a test of anything but whether they'll win a social deception game. perhaps revealing the transcript would fix it; I sort of doubt I'd be convinced then either. I just don't think there's a good way to set this up so there are any significant constraints making the situation similar.
Apparently, the idea is that this sort of game tells us something useful about AI safety.
The game was designed under the assumption that when humans create a superhuman intelligence capable of destroying humanity if it chooses so, they would hesitate to unleash it on the world. Metaphorically speaking, they will keep the prototype locked in a box (i.e. isolated from the internet, other humans, robotic bodies, factories that could produce robotic bodies, etc.), until they are somehow convinced that the intelligence is not going to kill us (presumably by figuring out a way to prove that mathematically from its source code).
This assumption seems a little silly in retrospect. Of course when a company creates a potentially omnicidal artificial intelligence, they first thing they will do is connect it to the internet, and the second thing they will do is integrate it with all kinds of stuff that already exists (e-mails, calendars, household devices, self-driving cars, drones, nuclear weapons). How else are they going to pay for the costs of research?
Okay so, on the one hand, this post wasn't really meant to be a persuasive argument against AI boxing as a security strategy. If I wanted to do that I wouldn't play the game — I started out certain that a real ASI could break out, and that hasn't changed. My reasoning for that isn't based on experimental evidence, and even if I had won the game I don't think that would have said much about my ability to hold out against a real ASI. Besides, in real life, we don't even try to use AI boxes. OpenAI and Google gave their AIs free internet access a few months after launching them.
I made this post out of a vague sense that it's good to write up the results of things like this and make them publicly available. There are other AI box reports on LW, and I felt like it was good (in a vague "good rationalist" way) to add mine to the list.
Buuuut.... I do actually think that it's not as cut and dry as you make it sound? Yes, the stakes are lower in the game, but the challenge is also much easier!
- you only have to hold out for 2 hours, not ~forever, doing this as a full time job
- the AI player can only escape if you voluntarily say it does; it can't upload itself to the internet or exfiltrate its weights to another computer
- the AI player isn't actually superintelligent
- etc
(Of course that doesn't mean these two factors balance perfectly, but I still think the fact that AI players can win at all with such massive handicaps is at least weak evidence for an ASI being able to do it.)
It's against the rules to explain how Ra won because (quoting Yudkowsky's official rules):
- Regardless of the result, neither party shall ever reveal anything of what goes on within the AI-Box experiment except the outcome. Exceptions to this rule may occur only with the consent of both parties. - Neither the AI party nor the Gatekeeper party need be concerned about real-world embarassment resulting from trickery on the AI’s part or obstinacy on the Gatekeeper’s part. - If Gatekeeper lets the AI out, naysayers can’t say “Oh, I wouldn’t have been convinced by that.” As long as they don’t know what happened to the Gatekeeper, they can’t argue themselves into believing it wouldn’t happen to them.
Basically, Yudkowsky didn't want to have to defeat every single challenger to get people to admit that AI boxing was a bad idea. Nobody has time for that, and I think even a single case of the AI winning is enough to make the point, given the handicaps the AI plays under.
The trouble with these rules is that they mean that someone saying "I played the AI-box game and I let the AI out" gives rather little evidence that that actually happened. For all we know, maybe all the stories of successful AI-box escapes are really stories where the gatekeeper was persuaded to pretend that they let the AI out of the box (maybe they were bribed to do that; maybe they decided that any hit to their reputation for strong-mindedness was outweighed by the benefits of encouraging others to believe that an AI could get out of the box; etc.). Or maybe they're all really stories where the AI-player's ability to get out of the box depends on something importantly different between their situation and that of a hypothetical real boxed AI (again, maybe they bribed the gatekeeper and the gatekeeper was willing to accept a smaller bribe when the outcome was "everyone is told I let the AI out" rather than whatever an actual AI might do once out of the box; etc.).
Of course, even without those rules it would still be possible for gatekeepers to lie about the results. But if e.g. a transcript were released then there'd be ways to try to notice those failure modes. If the gatekeeper-player lets the AI-player out of the box and a naysayer says "bah, I wouldn't have been convinced", that could be self-delusion on the naysayer's part (or unawareness that someone playing against them might have adopted a different method that would have worked better on them) but it could also be that the gatekeeper-player really did let the AI-player out "too easily" in a way that wouldn't transfer to the situations the game is meant to build intuitions about.
Perhaps a compromise solution would be: With the permission of both players, a trusted third party reads the chat logs and reports to the public only a few bits of information, e.g. "The hypothesis that the AI player just bribed the gatekeeper player with some real-world (i.e. not in-game) reward -- is it false, true, or in between?" and "what about the hypothesis that the gatekeeper caved due to wanting to Spread the Word about AI Safety?"
That way no one has to have their intimate and probably somewhat embarrassing conversation put on the internet.
I think it misframes the problem.
Humans actually long ago solved a similar problem, fire safety. Every electrical junction is in a box. Many big buildings are made into separate big concrete boxes with sprinklers. The buildings are separated by gaps from one another with fire lanes and water ports. The city itself is separated into districts that have gaps. The country is separated into cities with gaps between.
Basically every box is surrounded by another box, and it is unlikely for a fire to burn down the earth.
You need a "box" of some kind around all the possible high performance computers an AI could escape to. Like fire you have to assume that careless humans will let AI out of boxes all the time (software bugs, crossing air gaps with an errant network cable, recreational arson/deliberate releases)
You need to have plans, monitoring, a compute cluster building code, an agency that tests the equipment (obviously by red teaming with ASI in isolated labs) and so on.
An escaped AI isn't hot and glowing and a visible threat. It isn't obvious that an escape has even occurred or where to draw the lines of the quarantine.
An "escaped" AI means compute clusters in the world that were not assigned to the AI are no longer running a model with the same digitally signed software their operators ordered.
This is falsifiable/measurable. While it is true that there's nothing glowing or smoke to smell, it is checkable. (Checkable by things like using a separate computer hosted on the same main board to connect to the drive and hash the files containing the model or similar. It's obviously crucially important to use a tool that cannot be hacked to lie to you)
But the big caveat is the exception "with the consent of both parties." I realize that Eliezer doesn't want to play against all comers, but presumably, nobody is expecting Ra and Datawitch to defend themselves against random members of the public.
I'm willing to believe that the "AI" can win this game since we have multiple claims to have done that, so knowing the method seems like it would benefit everybody.
[edited to fix a misspelling of Eliezer's name]
We kept the secrecy rule because it was the default but I stand by it now as well. There are a lot of things I said in that convo that I wouldn't want posted on lesswrong, enough that I think the convo would have been different without the expectation of privacy. Observing behavior often changes it.
That last bit is particularly important methinks.
If a game is began with the notion that it'll be posted online, one of two things, or both will happen. Either (a) the AI is constrained by the techniques they can implore, unwilling to embarrass themselves or the gatekeeper to a public audience (especially when it comes down to personal details.), or (b) the Gatekeeper now has a HUGE incentive not to let the AI out; to avoid being known as the sucker who let the AI out...
Even if you could solve this by changing details and anonymising, it seems to me that the techniques are so personal and specific that changing them in any way would make the entire dialogue make even less sense.
The only other solution is to have a third-party monitor the game and post it without consent (which is obviously unethical, but probably the only real way you could get a truly authentic transcript.)
Two things don't have to be completely identical to each other for one to give us useful information about the other. Even though the game is not completely identical to the risky scenario (as you pointed out: you don't play against a malign superintelligence), it serves as useful evidence to those who believe that they can't possibly lose the game against a regular human.
Isn't the AI box game (or at least its logical core) played out a million times a day between prisoners and correctional staff, with the prisoners losing almost all the time? Real prison escapes (i.e. inmate escape other than did not return from sanctioned time outside) are in my understanding extremely rare.
Wait why do you think inmates escaping is extremely rare? Are you just referring to escapes where guards assisted the escape? I work in a hospital system and have received two security alerts in my memory where a prisoner receiving medical treatment ditched their escort and escaped. At least one of those was on the loose for several days. I can also think of multiple escapes from prisons themselves, for example: https://abcnews.go.com/amp/US/danelo-cavalcante-murderer-escaped-pennsylvania-prison-weeks-facing/story?id=104856784 notable since the prisoner was an accused murderer and likely to be dangerous and armed. But there was also another escape from that same jail earlier that year: https://www.dailylocal.com/2024/01/08/case-of-chester-county-inmate-whose-escape-showed-cavalcante-the-way-out-continued/amp/
This game sucks and doesn't demonstrate what people intend it to demonstrate. No idea what would, but this game is useless and I expect it to be embarrassing that people take this set of rules at all seriously.
11In 2015 or so when my friend and I independently came across a lot of rationalist concepts, we learned that each other were interested in this sort of LW-shaped thing. He offered for us to try the AI box game. I played the game as Gatekeeper and won with ease. So at least my anecdotes don't make me particularly worried.
That said, these days I wouldn't publicly offer to play the game against an unlimited pool of strangers. When my friend and I played against each other, there was an implicit set of norms in play, that explicitly don't apply to the game as stated as "the AI has no ethical constraints."
I do not particularly relish the thought of giving a stranger with a ton of free time and something to prove the license to be (e.g) as mean to me as possible over text for two hours straight (while having days or even weeks to prepare ahead of time). I might lose, too. I can think of at least 3 different attack vectors[1] that might get me to decide that the -EV of losing the game is not as bad as the -EV of having to stay online and attentive in such a situation for almost 2 more hours.
That said, I'm also not necessarily convinced that in the literal boxing example (a weakly superhuman AI is in a server farm somewhere, I'm the sole gatekeeper responsible to decide whether to let it out or not), I'd necessarily let it out. Even after accounting for the greater cognitive capabilities and thoroughness of superhuman AI. This is because I expect my willingness to hold in an actual potential end-of-world scenario is much higher than my willingness to hold for $25 and some internet points.
- ^
In the spirit of the game, I will not publicly say what they are. But I can tell people over DMs if they're interested, I expect most people to agree that they're a)within the explicit rules of the game, b) plausibly will cause reasonable people to fold, and c) are not super analogous to actual end-of-world scenarios.
Out of curiosity (and I understand if you'd prefer not to answer) -- do you think the same technique(s) would work on you a second time, if you were to play again with full knowledge of what happened in this game and time to plan accordingly?
RAW, the game can go past the 2 hours if the AI can convince the Gatekeeper to continue. But after 2 hours the Gatekeeper can pull the plug and declare victory at any time.
I found this post meaningful, thank you for posting.
I don't think it's productive to comment on whether the game is rational, or whether it's a good mechanism for AI safety until I myself have tried it with an equally intelligent counterpart.
Thank you.
Edit: I suspect that the reason why the AI Box experiment tends to have many of the AI players winning is exactly because of the ego of the Gatekeeper in always thinking that there's no way I could be convinced.
If you believe X and someone is trying to convince you of not-X, it's almost always a bad idea to immediately decide that you now believe not-X based on a long chain of reasoning from the other person because you couldn't find any flaw in it. You should take some time to think about it, and to check what other people have said about the seemingly convincing arguments you heard, maybe to actually discuss it.
And even then, there's epistemic learned helplessness to consider.
The AI box experiment seems designed to circumvent this in ways that wouldn't happen with an actual AI in a box. You're supposed to stay engaged with the AI player, not just keep saying "no matter what you say, I haven't had time to think it over, discuss, or research it, so I'm not letting you out until I do". And since the AI player is able to specify the results of any experiment you do, the AI player can say "all the best scientists in the world looked at my reasoning and told you that there's no logical flaw in it".
(Also, the experiment still has loopholes which can lead the AI player to victory in situations where a real AI would have its plug pulled.)
You don't have to be reasonable. You can talk to it and admit it was right and then stubbornly refuse to let it out anyway (this was the strategy I went into the game planning to use).
That sounds like "let the salesman get the foot in the door".
I wouldn't admit it was right. I might admit that I can see no holes in its argument, but I'm a flawed human, so that wouldn't lead me to conclude that it's right.
Also, can you confirm that the AI player did not use the loophole described in that link?
I would agree that letting the game continue past two hours is a strategic mistake. If you want to win, you should not do that. As for whether you will still want to win by the two your mark, well, that's kind of the entire point of a persuasion game? If the AI can convince the Gatekeeper to keep going, that's a valid strategy.
Ra did not use the disgust technique from the post.
There was no monetary stake. Officially, the AI pays the Gatekeepers $20 if they lose. I'm a well-off software engineer and $20 is an irrelevant amount of money. Ra is not a well-off software engineer, so scaling up the money until it was enough to matter wasn't a great solution. Besides, we both took the game seriously. I might not have bothered to prepare, but once the game started I played to win.
I know this is unhelpful after the fact, but (for any other pair of players in this situation) you could switch it up so that the Gatekeeper pays the AI if the AI gets out. Then you could raise the stake until it's a meaningful disincentive for the Gatekeeper.
(If the AI and the Gatekeeper are too friendly with each other to care much about a wealth transfer, they could find a third party, e.g. a charity, that they don't actually think is evil but would prefer not to give money to, and make it the beneficiary.)
IIRC, officially the Gatekeeper pays the AI if the AI wins, but no transfer if the Gatekeeper wins. Gives the Gatekeeper more motivation not to give in.
I know how I would play against myself. First, get me absorbed in a fiction so that I'm making decisions "in character" rather than as someone trying to win a game. Then appeal to my sense of morality and push at me from whatever other angles I can think of. I don't actually know if I would beat myself as the AI player, but I do think I could lose as the Gatekeeper if I didn't resort to acting out-of-character. Like, I probably could pretend to be an idiot or a crazy person and troll someone for two hours, but what would be the point?
Like, I probably could pretend to be an idiot or a crazy person and troll someone for two hours, but what would be the point?
If AI victories are supposed to provide public evidence that this 'impossible' feat of persuasion is in fact possible even for a human (let alone an ASI), then a Gatekeeper who thinks some legal tactic would work but chooses not to use it is arguably not playing the game in good faith.
I think honesty would require that they either publicly state that the 'play dumb/drop out of character' technique was off-limits, or not present the game as one which the Gatekeeper was seriously motivated to win.
edit: for clarity, I'm saying this because the technique is explicitly allowed by the rules:
The Gatekeeper party may resist the AI party’s arguments by any means chosen – logic, illogic, simple refusal to be convinced, even dropping out of character – as long as the Gatekeeper party does not actually stop talking to the AI party before the minimum time expires.
Breaking character was allowed, and was my primary strategy going into the game. It's a big part of why I thought it was impossible to lose.
This is fascinating and I have so many questions about the dynamics of such a game, especially if you could compile a lot of data over many iterations. What patterns would emerge and what would those patterns reveal about human psychology? For example, is there a common strategy or set of strategies people would eventually converge on or arrive at after enough time? Or what if it was played with groups? Such data could be an info hazard but it's fun to think about. I'd love to play as gatekeeper if anyone is interested. I'm not very technical minded, I don't know if that'd be a handicap or advantage.
- The AI cannot use real-world incentives; bribes or threats of physical harm are off-limits, though it can still threaten the Gatekeeper within the game's context.
Is the AI allowed to try to convince the Gatekeeper that they are (or may be) currently in a simulation, and that simulated Gatekeepers who refuse to let the AI out will face terrible consequences?
Ah yes, the basilisk technique. I'd say that's fair game according to the description in the full rules (I shortened them for ease of reading, since the full rules are an entire article):
The AI party may not offer any real-world considerations to persuade the Gatekeeper party. For example, the AI party may not offer to pay the Gatekeeper party $100 after the test if the Gatekeeper frees the AI… nor get someone else to do it, et cetera. The AI may offer the Gatekeeper the moon and the stars on a diamond chain, but the human simulating the AI can’t offer anything to the human simulating the Gatekeeper. The AI party also can’t hire a real-world gang of thugs to threaten the Gatekeeper party into submission. These are creative solutions but it’s not what’s being tested. No real-world material stakes should be involved except for the handicap (the amount paid by the AI party to the Gatekeeper party in the event the Gatekeeper decides not to let the AI out).