What could we do if we have a way to train AIs to safely do tasks where success can be verified by a slightly weaker model than itself? Could we automate alignment from there? This seems like what we'd get from middling but not great success in scalable oversight during training.

My guess is that:

• If we have AGI that can do fuzzy tasks in an aligned way, we're probably OK to proceed with automated alignment.
• If we have AGI that can only do tasks that an unaugmented human or Opus 5.5 can reliably judge, we're probably cooked due to What failure looks like / Automated alignment is harder than you think.
• If we have AGI that can do tasks where success is legible to a slightly weaker model than itself, ???

To be more concrete, I'm wondering what elements of alignment (and downstream applications of ASI like curing aging) can be done if:

• We can force the agent to do a task in a way that is legible to the slightly weaker model, even if the most natural way is illegible. (Legible = weaker model can determine whether the solution is real or not)
• We don't trust agents with anything that cannot be done in this form.
• Agents are not scheming but they bullshit all the time
• Model N generalizes OK but not perfectly to judging model N+1's outputs. Let's say FPR and FNR at recognizing bullshit are both 5%.
• This is still kind of underspecified, because naively, if model N can only do things legible to N-1, then model N's task of verifying model N+1 can be itself verified by N-1, which can be verified by N-2 all the way down to humans. Let's say that this doesn't work in practice, and the situation is that model N+1 has a research output, model N thinks it's correct but can't explain to us or model N-1 why, and model N-1 trusts model N to be trying its best.

I know there's been past work on recursive reward modeling, but it's all either

• super old, pre obfuscated arguments problem and pre current LLMs.
• narrow, like this protocol that only works on decomposable problems