Shortform

Cleo Nardo

This is a special post for quick takes by Cleo Nardo. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

Our abstractions in AI safety are so path dependent. It’s possible we went down the wrong path 10 years ago. I would like more people following entirely independent lineages of abstractions. Even though I think the main lineage (stuff downstream of Christiano/Irving) is modally likely to be correct.

8Thomas Kwa7d

Examples?

Hmm...

Train/eval/deploy. This abstraction was part of the "ML bundle" that Paul brought into AI safety in like 2018. Training is about building some model weight, evaluation is about deciding whether to deploy, and only deployment has real-world effects. So we worry that training-instances give us bad weights (gradient hacking, exploration hacking, alignment faking), or evaluation-instances give us bad decisions (sandbagging, alignment faking).
1. In practice, training and evaluation have side effects: agents hack out of the sandbox, or collude with each other... (read more)

5Mateusz Bagiński7d

It's also possible we went down the wrong path 80 years ago. (I'm singling this out as just one example of "it's not necessarily 'AI safety abstractions' but possibly a lot of baggage that has been inherited from prior currents of thought".)