Introduction

Why hasn't AI automated away more professions? One way of investigating this question is to compare the skills that real jobs require with current AI systems' capabilities. As long as there's a gap between these two, full automation is unfeasible.

One important limitation is that AI systems cannot yet act coherently over long periods of time. While AI has demonstrated impressive capabilities in solving concrete, isolated problems, maintaining consistent goals, reasoning, and plans over extended time frames has proven more challenging.

We know AI systems are getting better at completing tasks over longer time horizons, as METR’s recent work shows. But how much better will they need to get to start having real economic impact, or at least for their automation capabilities to increase significantly?

To answer this, we define sequential coherence as an AI system's ability to maintain consistent performance across extended task sequences. We estimate its importance as a bottleneck for automation by measuring the time needed to complete all remotable tasks in the O*NET dataset. Our findings suggest that AI agents could have enough sequential coherence to perform any remote economic task by 2030, and that AI automation potential could increase discontinuously.

What is Sequential Coherence?

Recent discussions about AI economics have highlighted that while LLMs are useful assistants that have increased worker productivity, their effects in terms of full-job automation have yet to be felt. The explanations offered usually invoke 'agency', 'autonomy', 'coherence', or 'adapt plans to circumstances'—concepts that are never clearly defined.

We define sequential coherence as the AI system's ability to maintain consistent performance quality across extended sequences of steps or interactions. The degree of sequential coherence required by a task is measured by the number of sequential steps or subtasks needed to complete it. When other factors are held constant, tasks requiring more steps demand higher sequential coherence. This explains why current AI systems excel at individual problems but struggle with complex, multi-step tasks—they hit a sequential coherence ceiling where performance degrades as step count increases, regardless of the difficulty of each individual step.

Illustrating Sequential Coherence

To illustrate our definition of sequential coherence...

Posts

Wikitag Contributions

Comments