ryan_greenblatt's Shortform

ryan_greenblatt

This is a special post for quick takes by ryan_greenblatt. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

It's wacky that we don't know if current AIs would try to take over the world if:

Takeover was easy
Takeover was an obvious way to accomplish an otherwise extremely difficult task
Takeover didn't have clearly evil vibes while clearly being takeover
The situation didn't look like a test

Checking is hard!

I suspect the answer is basically no, at least for AIs from Anthropic and OpenAI, but I'm not that confident. Models sometimes do crazy (misaligned) actions to try to succeed at tasks when they get stuck.

1Mikita Balesni6mo

Do you have some concrete operationalization of takeover that fits these requirements and happens while the model is deployed at a lab (e.g. in a Claude Code/Codex harness)? (Such that it would be possible to take some real traffic and edit it minimally to make a realistic version of this.)

No. But it seems tractable to have better tests of whether AIs would do intermediate bad actions using this methodology.

4Dagon6mo

Checking is hard! I don't think we even have a good operational definition of "take over" or any way to test for when/if it actually happens. that would seem a prerequisite for testing for intent. I think, for at least some definitions, the answer is obviously "yes". Given how hard coding assistants try to work around their own tool limitations, it's obvious that if "take over the world" was a shorter path (in their planner) to some goal, they'd take it.

ryan_greenblatt's Shortform

19

Ω 5

ryan_greenblatt's Shortform

19

Ω 5