To spell out this vibe-comic[1]: Safety made the locally correct decision here at every step of a bad situation. But like in Caleb Biddulph's parable, this globally resulted in an awkward state of affairs. From the Public's POV (penultimate panel), they would naively see AI safety as complicit in the...
(Edit: For some more recent views of Paul Christiano, see here and here. This post is meant as a "time-capsule" analysis, not to suggest he holds these exact opinions now.) A decade ago today, Paul Christiano published "Prosaic AI alignment" on his well-known AI safety blog. Alongside Concrete Problems in...
A lot of AI safety terms have nakedly anthropocentric definitions: * Reward "hacking" refers to pursuit of a high score on a task we gave, when it turns out we don't actually want a high score on that task[1] * Data "poisoning" refers to the insertion of training datasets that...
🔵 The blue team proposes an evaluation protocol for some capability/propensity of interest. This consists of a suite of measurement tasks, together with a preregistered decision-making process they will follow based on the results, e.g. what control measures they will put in place during deployment. 🔴 The red team proposes...
The untrusted monitoring protocol, as defined and evaluated in the AI control literature [1, 2, 3], looks like this: In comparison, the current monitoring setups at most frontier labs look like this: Before criticizing this, I first want to say that monitoring >> no monitoring. A year ago the 2nd...
The following are some further notes related to untrusted monitoring I had while working on our untrusted monitoring paper. The sections are mostly independent of each other. Views are my own. Untrusted-only Monitoring Doesn’t Work In some of our experiments we looked at the situation where the trusted monitor TM...