The recent METR report on AI acceleration at Anthropic has led to several analyses and arguments for rapid takeoff. There are important hard-to-supervise tasks that are necessary to make progress in AI research, and there are arguments that AI progress on these will lag progress on easily supervised tasks (like optimizing performance). The section "Why a Staged Model for Research Taste" presents an argument for this claim. I worked with Fable 5.2 to build a model where AI progress comes from a three-step process:
Figure 1. AI Research Model
Experiments produce results tested at small-scale, where judgment determines the quality of experiments
The best small-scale result is run at scale. transfer is the share of that gain that translates into experimental metrics at scale
"Real value" represents a hard-to-measure quantity (like model quality, innovation) that is the ultimate goal of the research, subject to a variable validity that determines how well the proxy metric transfers to real applications
Improvements in both transfer and validity are types of research taste. The iterative feedback loop makes transfer a supervisable task that is easier to optimize with AI (e.g., as argued by Dwarkesh Patel). The difficulty in measurement, catching gaming, and need to test in real-world experiments makes validity a hard-to-supervise fuzzy task that is hard to optimize with AI (e.g., as argued by Cindy Wu). Tom Reed's "Goodhart Singularity" argues that "the relevant data for superintelligence in most non-coding domains will only become available through deployment of AI models throughout the economy."
This model is derived from the AI Futures Project Q3.5 Timelines Update. The most salient difference is that my baseline scenario has the AI's research taste for validity increase by 0.5 SD for each year of progress at the 2025 pace, unlike its research taste for transfer that increases by the same 3 SD per year the AI Futures model has. Like that model, it uses a fixed projection of compute growth, at about 180% per year, and does not have it respond to progress (which seems reasonable over a ~2 year timeframe, absent a market crash).
The base model also adds a model of time allocation for researchers, who split their time among implementation, judging experiments, and overseeing agents. The section on pacing below proposes additional work for researchers to do assuming there is a requirement for human accountability and understanding.
Absent any pacing, this model predicts Top-Expert-Dominating-AI (TED-AI) in February 2029. If you adjust so validity improves at 3 SD per year of progress at the 2025 pace (as in the AI Futures model), it predicts TED-AI in June 2027, similar to AI Futures median prediction of July 2027. The acceleration in progress from compounding gains means that the takeoff just takes about a year and a half longer to get to TED-AI.
As Figure 2 below illustrates, the overall pace of progress plateaus or rises slowly until it takes off in late 2028 once AI ideas improve enough to outweigh its lagging validity judgment. Lab metrics of measured progress overstate real progress meaningfully during this period.
Figure 2. Apparent vs Real Progress
Timelines to TED-AI for varying parameter values:
AI judgment gained per year of progress
TED-AI
Ideas, transfer and metrics all +3 SD
June 2027
Ideas and transfer +3, metrics +1
Feb 2028
Ideas and transfer +3, metrics +0.5 (primary)
Feb 2029
Ideas and transfer +2.1, metrics +0.5
Sep 2029
Ideas +3, transfer and metrics +1
Jun 2028
Ideas +3, transfer +1, metrics +0.5
Jun 2029
Ideas +3, transfer and metrics +0.5
Feb 2030
Pacing Analysis
We model the impact of the following proposed pacing interventions that slow but do not pause capabilities research:
Internal-use lags. Limit capabilities of models used for AI R&D, so models can only be used for automated AI R&D 9 months after they are trained (option 3 in AI Futures How to Pace the US Frontier). Some have proposed allowing frontier models to be used only for safety research, not capabilities research.
Compute limits. Limit the compute allocated to capabilities research to be 9.6% of total compute owned by labs. The most restrictive versions of AI Futures option 2 limit capabilities to approximately this amount (for "safety min & inference min" they project that in addition to allocating 5% directly to capabilities, the safety work results in another 5% capabilities externality).
Comprehension audits. Requires frontier developers to demonstrate understanding of advances that are used. This is enforced by sampling reviews by embedded auditors to catch systematic failures (described in Pacing AI Automation Based on Understanding). This both requires researchers to spend more time understanding AI advances and limits advances that are deployed to those that can be understood by researchers (modeled as limiting ideas that can be used being those that a top human researcher could have proposed).
Audits and 9-month lag. This combines the first and third interventions: an internal-use lag of 9 months with comprehension audits to further pace and reduce risk.
The chart below shows how the various interventions delay the pace of progress. The interventions individually delay TED-AI by about a year, with the combination of audits and lagged use delaying by about a year and a half. While the time to reach TED-AI is similar, the trajectories are quite different. Internal-use lags and compute limits have the virtue of delaying the time until models get extremely capable, and they can meaningfully pace even if validity judgments improve quickly. By contrast, comprehension audits serve to moderate the pace of progress in the run up to TED-AI, with human understanding serving as a meaningful limit on runaway progress. Most of the pacing impact of comprehension audits comes from limiting ideas to those that human researchers can vouch for. If AI is able to explain advanced concepts effectively, the pacing effect would be diminished.
Figure 3 below shows how quickly AI progress is moving when TED-AI arrives under compute caps (~10x) and how quick it would be for a frontier model when it arrives under an internal use lag shown as a dotted orange line (44x). The lag delays TED-AI by about a year and it blocks the use of a far stronger model that could accelerate research far faster.
These pacing interventions can each also be tuned to further reduce risk. Internal-use lags could be set to allow use of recent models for safety research to differentially accelerate safety. Compute limits can be tuned to allow more compute to be used for safety research. And comprehension audits can reduce risks by requiring human accountability to verify important claims on which safety cases rest.
Combining an internal-use lag with requiring audits is robust to more scenarios and allows mitigating extreme rates of progress (limiting progress to 1.3x when TED-AI is achieved). The combination can be phased in: I believe internal-use lags can be mandated and enforced within months (although allowing safety carve-outs would add more complexity). Validating the approach and scaling comprehension audits will take more time but will provide additional pacing and ongoing mitigation of acceleration to justify the investment.
Figure 3. Comparing Pacing Interventions
Why a Staged Model for Research Taste
There has been a lot of debate recently about how quickly AI research taste will improve. In "Automated Alignment is Harder than You Think" Bowkis et al. posit that AI agents will lag in their ability to solve hard-to-supervise fuzzy tasks which they define as "tasks without clear evaluation criteria, for which human judgement is systematically flawed." Bowkis et al. say “Fuzzy tasks are particularly prevalent in alignment research compared to capabilities research.” They identify one such task that capabilities research shares: steering research and allocating resources between directions and note that “If an AI agent is trained on a systematically flawed human supervision signal for some research tasks, it will make mistakes when performing those research tasks”, and these are “errors that subsequent human review is unlikely to catch.” Wu has argued that such tasks are a meaningful constraint on capabilities research as well.
Benchmarks already show AI making progress on supervisable AI research tasks:
TasteVal consists of 8 novel tasks representative of frontier AI R&D, spanning pretraining data curation, pretraining and fine-tuning language models, preference modeling, and robustness to adversarial prompts. The authors note "Our tasks are fast and cheap to verify, which frontier labs find easiest to hill-climb" and it “doesn’t measure a model’s ability to choose which problems are most fruitful to work on." Frontier models are improving rapidly on it.
Anthropic's next-step study (in their RSI report) showed models making progress on individual steps in research where humans made mistakes. Models are showing rapid progress, with the best model beating humans 64% of the time if the human made a mistake, but only 20% of the time when the human took a good step.
RedwoodBench: "Models write 20 comments in response to conceptual AI safety google docs shared within Redwood Research (not published online). Models are graded on how many of the points from the human-written comments exist in the models’ comments"—summary by Hasan. RedwoodBench has 0.79 correlation with ECI (measured on 13 models). I believe this is highly correlated with pretraining, because the task is predicting what an expert would write.
LMCA: "Models evaluate arguments in conceptual domains, and are graded against expert ratings (AI safety, decision theory, normative ethics, philosophy of mind, politics)"—summary by Hasan. It has 0.89 correlation with Text Arena scores. I believe this is highly correlated with pretraining, because the task is predicting what an expert would write.
By contrast, current models have not made measurable progress on hard-to-supervise fuzzy tasks. Naturally, this is much harder to measure:
"Can AI agents conduct open-ended AI research? Early evidence from two case studies" (arXiv 2607.27191) and a follow up for GPT-6 Astra and Claude Fable 5.1 found poor end-to-end abilities (earning Reject or Strong Reject ratings).
Anthropic's TASTE benchmark shows current models performing essentially randomly with no correlation to ECI, even with Hasan's update using the most recent models. This is also a fairly limited test (92 pairs and human agreement on labels is only 77%). This was further analyzed recently by Ziqian Zhong, showing that when a shortcut is removed LLMs are getting worse at the task as ECI increases albeit on only 52 pairs.
Anthropic's "When AI Builds Itself" notes “In research, Claude can already match or outperform skilled humans at executing a well-specified experiment. However, large performance gaps persist when it comes to Claude exercising judgement in choosing goals in both engineering and research.”
Conclusion
This takeoff model explores how delays in AI progress in hard-to-supervise fuzzy tasks may affect timelines. It suggests that without pacing interventions, TED-AI would arrive in early 2029 as AI progress accelerates to a blistering pace. Given the risks involved in an explosive increase in AI capabilities, I believe that it will be important to pace automation, as in the combined interventions of lagging use of AI models along with requiring accountability by human researchers who understand research advances created by AI. The model predicts that this combination would slow progress and delay TED-AI until mid-2030.
Appendix
The model is calibrated to Anthropic. The AI Futures Project interpreted METR's report on acceleration at Anthropic as a pace of 1.2x in May 2026 and 1.7x in November 2026, compared to the 2025 pace. The model fits its pace on metrics to those numbers, resulting in real paces of 1.2x and 1.4x respectively. Anthropic's "How employment is changing at Anthropic" showed 19 agent hours per researcher hour in late August 2026.
I fitted only to Anthropic because the pace was measured there, and because its data suggest it is the furthest along in applying AI for research. It leads on ECI and shows acceleration above trend. Its agents do about five times as much work per researcher as OpenAI's do (their report extrapolates to about 3.8x by late August). Also, Anthropic reports a ~90% success rate for open-ended tasks without requiring corrections. OpenAI shows an 80% success rate without intervention for tasks that take humans 15 minutes and the same metric for Google is 11 minutes. The model follows one lab on one trajectory at central values. Nothing is sampled, so it gives dates and not ranges.
The 0.5 SD rate for validity is an assumption that is half of the low end of AI Futures' 80% interval. It is lower because of the evidence that this is a difficult area where models are not making quick progress. The model has no limit on how good ideas can be. Because the model predicts AI will be superhuman in earlier phases of research, it achieves TED-AI before the AI reaches the validity judgment of a median researcher.
The recent METR report on AI acceleration at Anthropic has led to several analyses and arguments for rapid takeoff. There are important hard-to-supervise tasks that are necessary to make progress in AI research, and there are arguments that AI progress on these will lag progress on easily supervised tasks (like optimizing performance). The section "Why a Staged Model for Research Taste" presents an argument for this claim. I worked with Fable 5.2 to build a model where AI progress comes from a three-step process:
Figure 1. AI Research Model
Improvements in both transfer and validity are types of research taste. The iterative feedback loop makes transfer a supervisable task that is easier to optimize with AI (e.g., as argued by Dwarkesh Patel). The difficulty in measurement, catching gaming, and need to test in real-world experiments makes validity a hard-to-supervise fuzzy task that is hard to optimize with AI (e.g., as argued by Cindy Wu). Tom Reed's "Goodhart Singularity" argues that "the relevant data for superintelligence in most non-coding domains will only become available through deployment of AI models throughout the economy."
This model is derived from the AI Futures Project Q3.5 Timelines Update. The most salient difference is that my baseline scenario has the AI's research taste for validity increase by 0.5 SD for each year of progress at the 2025 pace, unlike its research taste for transfer that increases by the same 3 SD per year the AI Futures model has. Like that model, it uses a fixed projection of compute growth, at about 180% per year, and does not have it respond to progress (which seems reasonable over a ~2 year timeframe, absent a market crash).
The base model also adds a model of time allocation for researchers, who split their time among implementation, judging experiments, and overseeing agents. The section on pacing below proposes additional work for researchers to do assuming there is a requirement for human accountability and understanding.
Absent any pacing, this model predicts Top-Expert-Dominating-AI (TED-AI) in February 2029. If you adjust so validity improves at 3 SD per year of progress at the 2025 pace (as in the AI Futures model), it predicts TED-AI in June 2027, similar to AI Futures median prediction of July 2027. The acceleration in progress from compounding gains means that the takeoff just takes about a year and a half longer to get to TED-AI.
As Figure 2 below illustrates, the overall pace of progress plateaus or rises slowly until it takes off in late 2028 once AI ideas improve enough to outweigh its lagging validity judgment. Lab metrics of measured progress overstate real progress meaningfully during this period.
Figure 2. Apparent vs Real Progress
Timelines to TED-AI for varying parameter values:
AI judgment gained per year of progress
TED-AI
Ideas, transfer and metrics all +3 SD
June 2027
Ideas and transfer +3, metrics +1
Feb 2028
Ideas and transfer +3, metrics +0.5 (primary)
Feb 2029
Ideas and transfer +2.1, metrics +0.5
Sep 2029
Ideas +3, transfer and metrics +1
Jun 2028
Ideas +3, transfer +1, metrics +0.5
Jun 2029
Ideas +3, transfer and metrics +0.5
Feb 2030
Pacing Analysis
We model the impact of the following proposed pacing interventions that slow but do not pause capabilities research:
The chart below shows how the various interventions delay the pace of progress. The interventions individually delay TED-AI by about a year, with the combination of audits and lagged use delaying by about a year and a half. While the time to reach TED-AI is similar, the trajectories are quite different. Internal-use lags and compute limits have the virtue of delaying the time until models get extremely capable, and they can meaningfully pace even if validity judgments improve quickly. By contrast, comprehension audits serve to moderate the pace of progress in the run up to TED-AI, with human understanding serving as a meaningful limit on runaway progress. Most of the pacing impact of comprehension audits comes from limiting ideas to those that human researchers can vouch for. If AI is able to explain advanced concepts effectively, the pacing effect would be diminished.
Figure 3 below shows how quickly AI progress is moving when TED-AI arrives under compute caps (~10x) and how quick it would be for a frontier model when it arrives under an internal use lag shown as a dotted orange line (44x). The lag delays TED-AI by about a year and it blocks the use of a far stronger model that could accelerate research far faster.
These pacing interventions can each also be tuned to further reduce risk. Internal-use lags could be set to allow use of recent models for safety research to differentially accelerate safety. Compute limits can be tuned to allow more compute to be used for safety research. And comprehension audits can reduce risks by requiring human accountability to verify important claims on which safety cases rest.
Combining an internal-use lag with requiring audits is robust to more scenarios and allows mitigating extreme rates of progress (limiting progress to 1.3x when TED-AI is achieved). The combination can be phased in: I believe internal-use lags can be mandated and enforced within months (although allowing safety carve-outs would add more complexity). Validating the approach and scaling comprehension audits will take more time but will provide additional pacing and ongoing mitigation of acceleration to justify the investment.
Figure 3. Comparing Pacing Interventions
Why a Staged Model for Research Taste
There has been a lot of debate recently about how quickly AI research taste will improve. In "Automated Alignment is Harder than You Think" Bowkis et al. posit that AI agents will lag in their ability to solve hard-to-supervise fuzzy tasks which they define as "tasks without clear evaluation criteria, for which human judgement is systematically flawed." Bowkis et al. say “Fuzzy tasks are particularly prevalent in alignment research compared to capabilities research.” They identify one such task that capabilities research shares: steering research and allocating resources between directions and note that “If an AI agent is trained on a systematically flawed human supervision signal for some research tasks, it will make mistakes when performing those research tasks”, and these are “errors that subsequent human review is unlikely to catch.” Wu has argued that such tasks are a meaningful constraint on capabilities research as well.
Benchmarks already show AI making progress on supervisable AI research tasks:
By contrast, current models have not made measurable progress on hard-to-supervise fuzzy tasks. Naturally, this is much harder to measure:
Conclusion
This takeoff model explores how delays in AI progress in hard-to-supervise fuzzy tasks may affect timelines. It suggests that without pacing interventions, TED-AI would arrive in early 2029 as AI progress accelerates to a blistering pace. Given the risks involved in an explosive increase in AI capabilities, I believe that it will be important to pace automation, as in the combined interventions of lagging use of AI models along with requiring accountability by human researchers who understand research advances created by AI. The model predicts that this combination would slow progress and delay TED-AI until mid-2030.
Appendix
The model is calibrated to Anthropic. The AI Futures Project interpreted METR's report on acceleration at Anthropic as a pace of 1.2x in May 2026 and 1.7x in November 2026, compared to the 2025 pace. The model fits its pace on metrics to those numbers, resulting in real paces of 1.2x and 1.4x respectively. Anthropic's "How employment is changing at Anthropic" showed 19 agent hours per researcher hour in late August 2026.
I fitted only to Anthropic because the pace was measured there, and because its data suggest it is the furthest along in applying AI for research. It leads on ECI and shows acceleration above trend. Its agents do about five times as much work per researcher as OpenAI's do (their report extrapolates to about 3.8x by late August). Also, Anthropic reports a ~90% success rate for open-ended tasks without requiring corrections. OpenAI shows an 80% success rate without intervention for tasks that take humans 15 minutes and the same metric for Google is 11 minutes. The model follows one lab on one trajectory at central values. Nothing is sampled, so it gives dates and not ranges.
The 0.5 SD rate for validity is an assumption that is half of the low end of AI Futures' 80% interval. It is lower because of the evidence that this is a difficult area where models are not making quick progress. The model has no limit on how good ideas can be. Because the model predicts AI will be superhuman in earlier phases of research, it achieves TED-AI before the AI reaches the validity judgment of a median researcher.
The model is open source and has more complete documentation at https://github.com/readback-ai/frontier-rd-taxonomy/tree/main/pacing-takeoff-model/chain