x

LESSWRONG

LW

frank_b_n — LessWrong

frank_b_n

frank_b_n

Message

15

4mo

frank_b_n

15

4mo

When does debate help a weak judge? Evidence from code and logic

by ethanelasky and frank_b_n

Authors: Ethan Elasky and Frank Nakasako, Palaestra Research; Naman Goyal, Independent. ArXiv link: [will be here when available] (for now, preprint Google Drive link) Thanks to Coefficient Giving for support and Thinking Machines for API credits; our mentor for guidance along the way; and Julian Michael, Johannes Gasteiger, and Jiaxin...

Inference-time Generative Debates on Coding and Reasoning Tasks for Scalable Oversight

by ethanelasky and frank_b_n

By Ethan Elasky and Frank Nakasako (equal contribution) We tested generative debate (where participants freely make their positions) on coding and reasoning tasks using weak-judge/strong-debater setups. The results are mostly negative: debate underperforms consultancy in 11 of 16 conditions, and removing debate transcripts entirely (just showing the judge the debater-generated...