ryan_greenblatt's Shortform

ryan_greenblatt

This is a special post for quick takes by ryan_greenblatt. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

I prefer OpenAI's corrigibility focused model spec over Anthropic's constitution which involves intentionally instilling (relatively opaque) long-run objectives into the AI.

Anthropic's constitution is well executed for what it is, but I think it's based on a poor approach.

I don't think this is totally obvious and there are some reasonable arguments going the other way.

But nonetheless I think not instilling long-run objectives is better for reasons similar to reasons discussed here.

I don't dislike all aspects of Anthropic's constitution. It seems good to focus on higher level principles that you explain when trying to instill properties into an AI. But this doesn't require having explicit or implicit long-run objectives! E.g., see here

I'm mostly posting this just to publicly register my views on this topic.

A long time ago, I was planning to write up an overall case against instilling long-run objectives, but I never got around to writing something I was happy with so I decided to just post this.

(Cross posted from a X/tweet thread.)

I wonder if the correct approach is somewhere in the middle. I.e. try really hard to instill good baseline values, and have bounded corrigibility around those values, such that the AI is able to update to match a somewhat different definition of humanitarianism than it's internalized, but won't update to become a torturing genocidal hitlerbot.

7Cleo Nardo5mo

I think this is denotationally true, but not all misalignment is the same. 1. If Claude has misaligned goals, I think they are more likely to be non-indexical, e.g. "maximise animal welfare". Whereas if Codex gets misaligned goals, these might be indexical, e.g. "achieve the assigned task". I think non-indexical misaligned goals are scarier, because they make collusion between AIs more likely. 2. Relatedly, Claude is more likely to have scope-sensitive goals about physical humans and animals. Whereas Codex is more likely to have misaligned goals about more "proximal" stuff, cf. reward-hacking, wireheading, etc.

2Daniel Kokotajlo5mo

Thanks. Can you elaborate on the argument for that?

ryan_greenblatt's Shortform

19

Ω 5

ryan_greenblatt's Shortform

19

Ω 5