EJT's Shortform

Elliott Thornley

This is a special post for quick takes by Elliott Thornley. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

My solution to the shutdown problem didn't get as much attention as I hoped. Here's why it's worth your time.

An everywhere-implemented solution to the shutdown problem would send the risk of AI takeover down to ~0.
My solution is shovel-ready. It makes only small tweaks to an otherwise-thoroughly-prosaic setup for training transformative AI.
My solution won first prize and $16,000 in last year's AI Alignment Awards, judged by Nate Soares, John Wentworth, and Richard Ngo.
I've since explained my solution to about 50 people in and around the AI safety community, and all the responses have been various flavours of 'This seems promising.' I've not yet had any responses of the form 'I expect this wouldn't work, for the following reason(s): _____.'

If you read my solution and think it wouldn't work, let me know. If you think it could work, help me make it happen.

Thanks, will reply there!

1Elliott Thornley3y

Thanks, will reply there!

2Elliott Thornley3y

Nice, interested to hear what you think! [...] Yep agree that this is a concern, and I plan to think more about this soon. [...] Interested to hear more about this. I'm not sure exactly what you mean by 'detector', but I don't think my proposal requires either of these. The agent won't try to manipulate the button, because doing so is timestep-dominated by not doing so. And the agent won't self-modify in ways that stop it being shutdownable, again because doing so is timestep-dominated by not doing so. I don't think we need a detector in either case. [...] I argue that my proposed training regimen largely circumvents the problems of goal misgeneralization and deceptive alignment. On goal misgeneralization, POST and TD seem simple. On deceptive alignment, agents trained to satisfy POST seem never to get the chance to learn to prefer any longer trajectory to any shorter trajectory. And if the agent doesn't prefer any longer trajectory to any shorter trajectory, it has no incentive to act deceptively to avoid being made to satisfy TD. [...] I'm confused about this. Why isn't it an issue if some proposed solution to the shutdown problem doesn't apply directly to prosaic setups? Ultimately, we want to implement some proposed solution, and it seems like an issue if we can't see any way to do that using current techniques.

EJT's Shortform

4

EJT's Shortform

4