This is a special post for quick takes by leogao. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
which of the following is more important to solve for alignment?
having the ability to determine whatsoever whether any given action is dangerous (but this could be extremely expensive per action)
cheaply determining whether a given action is dangerous, given access to an oracle is always correct about whether it's dangerous (but the oracle can be extremely expensive)
what counts as an action? if you mean tight definitions e.g. every toolcall is an action then I mostly agree with @TsviBT. you can have a dangerous rollout where no individual action passes any danger threshold, and for notkilleveryoneism risks this is plausibly the default, since long-term plans are diffuse. e.g. an oracle on the legality of official acts wouldn't necessarily have stopped hitler
10TsviBT
Like, you're just watching the externally visible actions of the agent? I think this would in fact not really pin down the answer, except to, like, Solomonoff induction (basically reading / guessing the internals of the agent by induction, and then evaluating those internals). There's definitely no such thing as a "dangerous action" in this context, in the relevant sense. I mean, your question kinda makes sense if you ask it like "Is it more important to go from no judge of alignment to an expensive judge, or to go from an expensive judge to a feasible judge?". But the thing about actions makes the question not make sense as stated.