x

LESSWRONG

LW

f_niv — LessWrong

f_niv

f_niv

Message

20

1

6mo

f_niv

20

6mo

When does debate help a weak judge? Evidence from code, logic and math

by ethanelasky and f_niv

2026.06.07 update: We’ve rewritten this entire post for clarity and added a new math setting to bolster our results. Authors: Ethan Elasky and Frank Nakasako, Palaestra Research; Naman Goyal. Link to ArXiv preprint Thanks to Coefficient Giving for support and Thinking Machines for API credits; our mentor for guidance along...

Inference-time Generative Debates on Coding and Reasoning Tasks for Scalable Oversight

by ethanelasky and f_niv

By Ethan Elasky and Frank Nakasako (equal contribution) We tested generative debate (where participants freely make their positions) on coding and reasoning tasks using weak-judge/strong-debater setups. The results are mostly negative: debate underperforms consultancy in 11 of 16 conditions, and removing debate transcripts entirely (just showing the judge the debater-generated...