LESSWRONG
LW

Value Learning

Oct 29, 2018

by rohinmshah

This is a sequence investigating the feasibility of one approach to AI alignment: value learning.

Preface to the sequence on value learning

64
rohinmshah
4moΩ 18Show Highlight
6

Ambitious Value Learning

What is ambitious value learning?

44
rohinmshah
4moΩ 12Show Highlight
28

The easy goal inference problem is still hard

38
paulfchristiano
4moΩ 9Show Highlight
16

Humans can be assigned any values whatsoever…

43
Stuart_Armstrong
4moΩ 11Show Highlight
8

Latent Variables and Model Mis-Specification

19
jsteinhardt
4moΩ 5Show Highlight
1

Model Mis-specification and Inverse Reinforcement Learning

30
Owain_Evans, jsteinhardt
4moΩ 7Show Highlight
1

Future directions for ambitious value learning

42
rohinmshah
4moΩ 12Show Highlight
9

Goals vs Utility Functions

Ambitious value learning aims to give the AI the correct utility function to avoid catastrophe. Given its difficulty, we revisit the arguments for utility functions in the first place.

Intuitions about goal-directed behavior

29
rohinmshah
3moΩ 7Show Highlight
12

Coherence arguments do not imply goal-directed behavior

62
rohinmshah
3mo7 min readΩ 19Show Highlight
22

Will humans build goal-directed agents?

39
rohinmshah
2moΩ 11Show Highlight
41

AI safety without goal-directed behavior

40
rohinmshah
2moΩ 12Show Highlight
9

Narrow Value Learning

What is narrow value learning?

19
rohinmshah
2moΩ 6Show Highlight
3

Ambitious vs. narrow value learning

18
paulfchristiano
2moΩ 6Show Highlight
15

Human-AI Interaction

18
rohinmshah
2moΩ 7Show Highlight
7

Reward uncertainty

18
rohinmshah
1moΩ 6Show Highlight
0

The human side of interaction

16
rohinmshah
1moΩ 6Show Highlight
2

Following human norms

23
rohinmshah
1moΩ 9Show Highlight
8

Future directions for narrow value learning

12
rohinmshah
1moΩ 7Show Highlight
4

Conclusion to the sequence on value learning

44
rohinmshah
1moΩ 13Show Highlight
13