657

LESSWRONG
LW

656
AI
Frontpage

4

[ Question ]

What is "Instrumental Corrigibility"?

by joebernstein
23rd Jun 2020
1 min read
A
1
1

4

AI
Frontpage

4

What is "Instrumental Corrigibility"?
2Gurkenglas
New Answer
New Comment

1 Answers sorted by
top scoring

Gurkenglas

Jun 23, 2020

20

An instrumentally corrigible agent lets you correct it because it expects you know better than it. The smarter it becomes, the less your higher competence is worth, and the more it loses out by letting you take the wheel while you're not perfectly aligned with it.

Add Comment
Moderation Log
More from joebernstein
View more
Curated and popular this week
A
1
0

In his article here, Paul Christiano mentions the idea that instrumentally corrigible agents are not robust, as slight misalignments between their values and the overseer values would lead to an adversarial relationship. While the intuition that slight deviations in values potentially results in adversarial behavior makes sense, I am not sure if I quite understand the meaning behind what an instrumentally corrigible agent is.

I would presume it relates to how corrigibility arises, that is different from Paul's amplification approach.