TL;DR

There is some consensus that LLMs are bad at hard-to-verify tasks. The question is whether models are getting better at them over time. As a motivating example, I gave one research-reproduction task to 12 models spanning three years of progress, to illustrate how (1) what looked like an emergent capability was a predictable trend, visible years earlier if progress was measured with granularity, and (2) for the earliest models, building a verifiable check would have been close to impossible: their attempts were so far from correct that a check would have had nothing to grade, so the task itself would have looked unverifiable. When thinking about current capabilities or forecasting future ones, researchers should be aware that binary judgments of usefulness can hide steady partial progress, and that a task looking 'unverifiable' today may say as much about the current capability profile of models as about the task itself.

Introduction

Frontier models like Fable seem to struggle with novel end-to-end research [1, 2], often producing slop, sometimes slop so bad that it would get you banned from arXiv [3]. Meanwhile, SWEs and researchers seem to find these tools incredibly useful. The best models are now capable of impressive feats of engineering, like migrating Bun to Rust in a week [4] or replicating full programs that took years to create [5].

This gap in capabilities has been documented before [6, 7, 8], and is sometimes referred to as easy- vs hard-to-verify tasks. The standard explanation is that RLVR favors progress on tasks where good and bad outputs are easy to tell apart.

Especially when talking to other researchers, this gap is sometimes presented as evidence for longer timelines to an AI intern.

A common version of the argument goes:

Research taste has incredibly long feedback loops. Even peer review is often not enough to know how impactful a research idea will be; the only judge is time and impact in the real world. And hence "fancy tricks" with LLM judges, human preference, or simulated research universes will not suffice.

We have two ways of forming an opinion about progress in these domains: we can use our own judgment, formed by working with these models on a daily basis, or we can develop targeted evaluations. Creating evals for hard-to-verify tasks is notoriously hard, and making them scalable requires tradeoffs like using imperfect LLM judges ...

Robyn Kozierok

Robyn Kozierok