TL;DR - The benchmarks frontier labs cite for long-horizon capability mostly measure capability on tasks involving many steps or lots of context, verifiable end states, and little external input. I argue that what defines real-world long-horizon work is delayed external feedback. That delay is where human input becomes high leverage,...
This is a crosspost from our report website for Why LLMs Aren't Scientists Yet: Lessons from Four Autonomous Research Attempts. This report details the work behind our LLM-written paper "The Consistency Confound: Why Stronger Alignment Can Break Black-Box Jailbreak Detection" accepted at Agents4Science 2025, the first scientific conference requiring AI...
TL;DR - Adding “for AI safety research” increased refusals on a harmless paraphrasing task for some top models. Conservative, aligned public models might be less useful for AI safety research by default. I ran a quick, contained experiment while building a benign, context-matched split of HarmBench (proposed by Mazeika et...