Frontier coding agents claim work they didn't do
by NolanSmyth, Yorguin-José Mantilla-Ramos, Pascal Jr Tikeng Notsawo, Saskia Helbling, Alberto Tosato, Mohamed Amine Merzouk, Nouha DZIRI, Gauthier Gidel, and Tommaso (Tommie) Tosato
Tara Research Team arXiv | Twitter thread tl;dr. We built OverclaimBench, which asks frontier coding agents to review a corpus of code or text documents, then compares their final report with their transcripts. Pooled across 12 frontier agents and five scenarios, two-thirds of runs didn't open some of the files...
Nov 56