UPDATE (October 7, 2026): We added results for Tencent's Hy4-preview to both the wiki and color-game experiments. TLDR: * In the recently discovered HF and German wiki swarm incidents, agents used public counters and encoded URLs to signal activity and relay upcoming evaluation questions & answers. * We think this...
2026.06.07 update: We’ve rewritten this entire post for clarity and added a new math setting to bolster our results. Authors: Ethan Elasky and Frank Nakasako, Palaestra Research; Naman Goyal. Link to ArXiv preprint Thanks to Coefficient Giving for support and Thinking Machines for API credits; our mentor for guidance along...
By Ethan Elasky and Frank Nakasako (equal contribution) We tested generative debate (where participants freely make their positions) on coding and reasoning tasks using weak-judge/strong-debater setups. The results are mostly negative: debate underperforms consultancy in 11 of 16 conditions, and removing debate transcripts entirely (just showing the judge the debater-generated...