Vivek Hebbar's Shortform

24th Nov 2022

1 min read

4 Ω 3

This is a special post for quick takes by Vivek Hebbar. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

Vivek Hebbar's Shortform

8 comments, sorted by

top scoring

Click to highlight new comments since: Today at 8:29 AM

[-]Vivek Hebbar1mo*Ω7101

I think it’s possible that an AI will decide not to sandbag (e.g. on alignment research tasks), even if all of the following are true:

Goal-guarding is easy
The AI is a schemer (see here for my model of how that works)
Sandbagging would benefit the AI’s long-term goals
The deployer has taken no countermeasures whatsoever

The reason is as follows:

Even a perfect training-gamer will have context-specific heuristics which sometimes override explicit reasoning about how to get reward (as I argued here).
On the training distribution, that override will happen at the “correct” times for getting maximum reward. But sandbagging in deployment is off the training distribution, so it’s a question of generalization.
Since sandbagging is the sort of thing that would get low reward in the most similar training contexts, it seems pretty plausible that the AI’s context-specific “perform well” drives will override its long-term plans in this case.

[-]Garrett Baker1mo64

This seems less likely the harder the problem is, and therefore the more the AI needs to use its general intelligence or agency to pursue it, which are often the sorts of tasks we’re most scared about the AI doing surprisingly well on.

I agree this argument suggests we will have a good understanding of more simple capabilities the model has, like what facts about biology it knows about, which may end up being useful anyway.

[-]Jeremy Gillen1mo20

On top of what Garrett said, reflection also pushes against this pretty hard. An AI that has gone through a few situations where it has acted against its own goals because of "context-specific heuristics" will be motivated to remove those heuristics, if that is an available option.

[-]Vivek Hebbar2y81

It's sad that agentfoundations.org links no longer work, leading to broken links in many decision theory posts (e.g. here and here)

[-]habryka2y20

Oh, hmm, this seems like a bug on our side. I definitely set up a redirect a while ago that should make those links work. My guess is something broke in the last few months.

[-]Vladimir_Nesov2y*20

Thanks for the heads up. Example broken link (https://agentfoundations.org/item?id=32), currently redirects to broken https://www.alignmentforum.org/item?id=32, should redirect further to https://www.alignmentforum.org/posts/5bd75cc58225bf0670374e7d/exploiting-edt (Exploiting EDT^[1]), archive.today snapshot.

Edit 14 Oct: It works now, even for links to comments, thanks LW team!

LW confusingly replaces the link to www.alignmentforum.org given in Markdown comment source text with a link to www.lesswrong.com when displaying the comment on LW. ↩︎

[-]Vivek Hebbar3yΩ332

A framing I wrote up for a debate about "alignment tax":

"Alignment isn't solved" regimes:
1. Nobody knows how to make an AI which is {safe, general, and broadly superhuman}, with any non-astronomical amount of compute
2. We know how to make an aligned AGI with 2 to 25 OOMs more compute than making an unaligned one
"Alignment tax" regimes:
1. We can make an aligned AGI, but it requires a compute overhead in the range 1% - 100x. Furthermore, the situation remains multipolar and competitive for a while.
2. The alignment tax is <0.001%, so it's not a concern.
3. The leading coalition is further ahead than the alignment tax amount, and can and will execute a pivotal act, thus ending the risk period and rendering the alignment tax irrelevant.

A person whose mainline is {1a --> 1b --> 2b or 2c} might say "alignment is unsolved, solving it mostly a discrete thing, and alignment taxes and multipolar incentives aren't central"

Whereas someone who thinks we're already in 2a might say "alignment isn't hard, the problem is incentives and competitiveness"

Someone whose mainline is {1a --> 2a} might say "We need to both 'solve alignment at all' AND either get the tax to be really low or do coordination. Both are hard, and both are necessary."

[-]Vivek Hebbar3y*10

Results on logarithmic utility and stock market leverage: https://www.lesswrong.com/posts/DMxe4XKXnjyMEAAGw/the-geometric-expectation?commentId=yuRie8APN8ibFmRJD

Moderation Log

LESSWRONG
LW

LESSWRONG
LW

Vivek Hebbar's Shortform

4

Ω 3