METR: Independent Assessment of AI R&D Acceleration at Anthropic in October 2026

Zach Stein-Perlman

METR: Independent Assessment of AI R&D Acceleration at Anthropic in October 2026 — LessWrong

168 METR: Independent Assessment of AI R&D Acceleration at Anthropic in October 2026

by Zach Stein-Perlman

5th Nov 2026

AI Alignment ForumLinkpost for metr.org

4 min read

168 Ω 46

Executive summary
From August 12 to October 29, 2026, METR staff performed a high-access evaluation at Anthropic to assess the degree to which AI is accelerating AI R&D. If AI systems started to significantly accelerate further AI development, this could lead to a sudden emergence of extreme and risky capabilities.
Anthropic offered unprecedented access to internal data on their AI R&D process, including a variety of metrics on pretraining, reinforcement learning (RL), training compute in each domain, benchmark scores for internal models, AI speedup analyses, forecasts of a future model’s metrics, and interviews with relevant staff.
In our first section, we share our subjective conclusions about risk. We now believe that:
As of October 2026, Anthropic is likely improving the capabilities of their models on well-specified tasks at around twice the rate they did between March 2024 and October 2025. This estimate is given with high uncertainty; we would not be surprised if the true acceleration was closer to 1.3x or 3x the historical rate.
A majority of this acceleration today is likely attributable to Anthropic’s extensive use of AI to do R&D, though there is significant uncertainty and other causes likely played a large role in acceleration in late 2025.
The lack of observed bottlenecks or strong diminishing returns means there is a significant risk that further capabilities progress could lead to even faster acceleration, though, as we say below, we may be missing important bottlenecks.
In our second section we share some of the key empirical findings from our Full Report:
Since October 2025, benchmark scores (measured via AECI, Anthropic’s version of the Epoch Capabilities Index) for Anthropic’s models have been growing at an estimated 1.9x the pre-October 2025 rate [95% confidence interval: 1.5–2.8x], implying that Anthropic has made roughly two years of historical progress over the last year.
We are confident that the acceleration was not due to an acceleration in the scaling of training compute.
Interviews and surveys are consistent with AI usage causing a doubling in the pace of capabilities progress. Anthropic staff self-report dramatic uplift from use of AI systems in interviews and surveys, although these are challenging to interpret. We were told that R&D is accelerated primarily via AI assistance rather than fully autonomous contributions, but there are some signs that fully autonomous contributions will play a larger role soon.
We did not see evidence for significant bottlenecks which would prevent further capabilities progress due to limits on algorithmic progress, training compute, experiment compute, or data in the next 12 months. However, our understanding of the status of different bottlenecks changed several times as we received more detailed information, so we are not confident in our current assessment.
Some of the key findings that most informed our conclusions about risk were redacted by Anthropic.
In our third section, we describe the assessment process:
The scope, setup, and timeline of this assessment.
Details of the information shared with METR.
Details of what information and evidence we were not able to share given Anthropic’s redactions.
Finally, we describe the biggest limitations of our analysis:
We could be underestimating capability acceleration if we did not assign enough importance to (1) evidence that some growth rates have further increased recently; (2) further automation of AI R&D tasks leading to a further acceleration in progress; (3) evidence that AECI growth has sometimes understated growth in real-world impact.
We could be overestimating capability acceleration if we did not assign enough importance to (1) non-AI factors in Anthropic’s research progress; (2) AECI’s focus on verifiable tasks; (3) reasons why Anthropic’s early growth rate (2023–2025) was unusually low.
The remainder of the report goes into detailed discussion of each of the key components. We include appendices with supplemental material.

Twitter thread.

Ryan Greenblatt:

This report on AI R&D automation (RSI) is a big update towards ASI within 1-2 years. It says AI progress is sped up by RSI right now by around 2x. In AI 2027, this happens around March 2027.
I was seriously wrong about RSI speed up; I previously expected much less in 2026 (around 1.15x-1.3x). It seems as though there was specific privately available evidence that would have changed my mind.
I haven't seen an unredacted version of this report; this is purely about the public report. Because so much of the underlying evidence was redacted from the public report, I think conclusions about things other than the observed 2x faster rate of progress on benchmarks depend significantly on trust for the authors of the report. (E.g., conclusions about whether this speed up is from RSI depend on trust.) I do trust the authors of the report a lot to be careful and rigorous and generally do a good job. (Note that I also work at METR.) Less extensive redactions would greatly reduce this need for trust.

Ajeya Cotra:

Our colleagues found that Anthropic’s capabilities progress has doubled, mainly due to AI. Ryan’s modeling from public info suggests the whole company’s R&D would be going 4-5x faster. Labor uplift would be higher still. This is a major update to very short timelines.
I am sad that so much of the underlying evidence in this report was redacted. I think evidence transparency is very important for the world to figure this out in time. Evidence about risk should be transparent.

Daniel Kokotajlo: says the METR report confirms his worry from two days ago, which was:

We at AIFP are spooked by [Anthropic's S1 update]. IF (big IF) Anthropic really is imminently about to meet the RSP threshold of 2x acceleration over baseline, then it seems timelines to full RSI and superintelligence may be very short indeed. For example if we naively optimize the AIFM parameters to achieve 2x acceleration over baseline progress in Jan 2027, we get superintelligence sooner than depicted in AI 2027.

168 Ω 46

New Comment

43 comments, sorted by

top scoring

Click to highlight new comments since: Today at 12:41 AM

[-]Thane Ruthenis2d40-3

As of October 2026, benchmark scores appear to be growing at 1.9x their pre-October 2025 rate

... isn't this graph actually kind of a bear signal?

No, seriously. The hypothesis we're tracking is RSI, i. e., that AI capability improvements make additional AI capability improvements easier. This should mean that the acceleration multiplier is growing: from one model release to another, the slope of the graph should be becoming ever steeper, such that the graph as a whole stops being well-modeled as linear.

What the graph is showing, instead, is a one-off slope change somewhere around the start of 2026, past which the slope remained the same. That is, there was some sort of "unlock" in early 2026 that let AI make Anthropic employees more productive^[1], and then additional capability improvements no longer made further capability improvements easier. For the entire year.

I suppose the bull case is that RSI is just "discrete" instead of continuous? I. e., that there will be a few more discrete slope changes like this, and each slope change will bring the next slope change sooner, up until AI is able to handle all research tasks and we get a multiplier of "approximately positive infinity" or something. But this isn't how the argument usually goes, I think.

Edit: Employees' self-reported uplift is more like how RSI should look like, a straight line on the productivity multiplier graph. But these data are considered much more unreliable, and if it's to be taken seriously together with the other graph, it would imply that sustaining the current 2x growth would require continual exponential improvement in employee productivity.

^{^}
E. g., the "now only bottlenecked on ideas & compute, no longer bottlenecked on code" stuff from the interviews.

[-]Julian Bradshaw2d*81

You're right that an intelligence explosion^[1] would require a growing acceleration multiplier (for some period of time). But I don't think seeing stable-ish acceleration with a major trend breakpoint today is bearish. The AI Futures Model predicts that we see low acceleration from ~2024 up until we get an Automated Coder in [Daniel: Oct '27, Eli: Jun '28, Brendan: Jan '28], and then you get noticeably growing acceleration in takeoff toward ASI (eventually leveling off). For example, here's what Eli Lifland's presets were predicting as of 8/16/26 for ECI scores:^[2]

You can see here that the trendline from 2024 to Oct. 2025, and then from Oct. 2025 to Nov. 2026, is basically supposed to be just a straight line (aka, very low acceleration). But if you zoomed in or looked closely at the data, you'd find that technically there is some acceleration, up to ~1.08x by Nov. 2026. The acceleration only noticeably grows after AC, when the trendline suddenly steepens in takeoff.

So that's why the AIFP (and Ryan Greenblatt, who is closely associated) is so "spooked". Acceleration in capabilities growth wasn't supposed to be this strong yet!

Now for various reasons I think the headline 1.9x acceleration is overstated relative to the "true" frontier AI capabilities acceleration that the AIFM tries to predict. But it does seem like this evidence from METR ought to move timelines forward meaningfully on the most rigorous model we have.^[3]

^{^}
You can have mere Recursive Self-Improvement (RSI) without an Intelligence Explosion, and even reach ASI eventually that way. RSI like you're talking about, where "AI capability improvements make additional AI capability improvements easier", only requires an acceleration multiplier that stays above 1. Though I kind of prefer definitions of RSI that just mean "AI that improves itself" even if capability improvements become harder.
^{^}
And this single-run-on-presets chart is more aggressive than his "true opinion" as of 8/16/26. Per here, his all-things-considered Monte Carlo-based forecast puts p50 for AC at Jan '30! But on the other hand, this graph is more conservative than that for Daniel Kokotajlo's presets, where ASI hits earlier and on the steepest part of the curve.
^{^}
Some initial modeling I don't trust suggests that taking the 1.4x acceleration today's METR report gives for the all-labs frontier as true moves ASI 8 to 14 months earlier in the AIFM across different presets. Need to think more about how to feed this evidence into the model and do Monte Carlo runs though. Not sure if I'll have time to do that before AIFP puts out their update. EDIT: doing three 1000-draw Monte Carlo runs (two different approaches, one with two different parameter sets) I get median ASI date shifting anywhere from 2 to 41 months earlier (yes this is physically possible b/c Eli's Monte Carlo median is in 2036 due to a long right tail). Definitely needs more rigorous thought.

[-]Thane Ruthenis1d*2010

So that's why the AIFP (and Ryan Greenblatt, who is closely associated) is so "spooked". Acceleration in capabilities growth wasn't supposed to be this strong yet!

Sure, but isn't such a significant AI-driven speedup in capabilities growth without compounding multiplicative dynamics itself an unexpected model violation? Yes, the speed-up wasn't supposed to be that big yet, but at the point where it's that big it's also supposed to have already made RSI very visibly kick in. And yet it (it appears) didn't.

[-]Julian Bradshaw1d51

That's a clear framing of your stance, thanks. And a good objection! Here's my thoughts:

Yeah it's unexpected by AIFM on default presets that we'd see >~1.3x frontier ECI acceleration prior to the Automated Coder (AC). But the 1.9x is Anthropic-only AECI acceleration; the frontier AECI acceleration from the METR report is 1.4x, pretty close, and even that might be less reflective of reality than the lower ECI acceleration trends.^[1]

Also, in a certain sense the main chart in this METR report isn't even trying to model the chance of a growing acceleration multiplier. There's suggestive evidence that this does exist, ex. the fact that AECI-estimated acceleration multiplier has kept increasing over the past couple months, and ex. that METR says in their report today that, per redacted evidence, acceleration could be increasing over time. But the main AECI trend modeling is explicitly focused on the frame where there's a single breakpoint in the trendline:

We estimate a learned breakpoint in the trend, though we exogenously impose that there is 1 breakpoint.

In other words, the AECI data is mostly reflective of a constant acceleration multiplier, but there's other reasons to think there's a growing acceleration multiplier.^[2]

^{^}
More of my reasoning on this at bottom of this comment.
^{^}
METR also holds this stance. In Limitations, they say:
We may have missed recent further acceleration. The rate of growth could be undergoing continuous acceleration, e.g. hyperbolic growth. While we do not see clear evidence of this in the AECI trends, there is other evidence that makes this seem more plausible which Anthropic redacted. If growth is hyperbolic, we are likely underestimating instantaneous acceleration and substantially understating acceleration in the near future. In addition, we observe model-level outcomes only after a model is trained, meaning we observe acceleration in effective R&D with a lag roughly equal to the model’s training time.

[-]Thane Ruthenis1d40

Also, in a certain sense the main chart in this METR report isn't even trying to model the chance of a growing acceleration multiplier

Sure, but just eyeballing it, it does seem that a 2-piece linear is indeed a good fit? I don't think an exponential/hyperbolic would fit that graph well. So it's a property of these specific underlying data, not of the modeling choice.

the fact that AECI-estimated acceleration multiplier has kept increasing over the past couple months

I've seen this, yup. But as you describe it, it's "updating this estimate of AI capabilities acceleration upward", as in it's a change in estimate over the same time period, caused by re-doing the estimate on different version of AECI (or something). And, again, none of the fixed-methodology graphs actually seem to fit an exponential/hyperbolic well.

METR also holds this stance

I'd missed that line, see the edit there.

[-]Julian Bradshaw18h60

Sure, but just eyeballing it, it does seem that a 2-piece linear is indeed a good fit? I don't think an exponential/hyperbolic would fit that graph well. So it's a property of these specific underlying data, not of the modeling choice.

Yes I agree that it seems like a good fit to the AECI data. METR also says they "do not see clear evidence of this in the AECI trends" where "this" is hyperbolic trends. But they don't actually put any detailed evidence of attempted fits into their report, they just work with the linear + breakpoint regime throughout.

It's also important to note that the AECI trendline post-Oct-2025-breakpoint here has only 10 points of data, of which 7 are distinct models, and 3 of the points fall within a 2.5 week period in Sep 2026. So a gentle hyperbolic curve is hard to distinguish from a linear slope with this amount of data, esp. considering the data points keep shifting slightly every few weeks.

Doing some analysis on if, given 10 data points like these, how often we could detect a "true" underlying slight hyperbolic trend:

...okay this actually surprised me. The post-break data, at best fit, shows a slight decrease in "acceleration over pre-break data"^[1], basically due to how strong Mythos Preview was. But of course it's all underpowered, and I'm not entirely sure if my approach above is reasonable. We also don't quite have the data from METR/Anthropic to do a great job of trying out hyperbolic fits here, but doing the best I can, the data says the slope at Sep 2026 is 0.54x to 1.25x (95% CI) the Oct-2025 breakpoint starting slope. Feel free to steal my chart on that haha, but maybe verify it's sensible first.

---

But as you describe it, it's "updating this estimate of AI capabilities acceleration upward", as in it's a change in estimate over the same time period, caused by re-doing the estimate on different version of AECI (or something).

This is getting into the weeds but my best estimate is that the rise from 1.53x for Opus 5.5's system card to 1.90x in METR's data is +0.21x from METR's differences in method + AECI reruns and +0.16x from adding the internal models. Also technically the time period changes slightly each time, that's part of the difference in METR's method. The increases across the Opus/Sonnet/Haiku system cards do come mostly from AECI reruns.

But to explain why I think this rising estimate is suggestive evidence toward growing acceleration:

We went from the evidence indicating "it was a one-time jump at Mythos Preview" to "oh there was some constant acceleration" to "oh there was a bunch of constant acceleration" and now METR is saying (paraphrased) "we can't really rule out that actually the acceleration is growing, and we have some redacted evidence indicating this, though AECI doesn't necessarily indicate it". So I'm expecting (with some probability) evidence in the future that will update me toward growing acceleration.

In other words, it seems reasonable to put some probability on the idea that, as our measurements of past progress improve, it will in fact turn out to be the case that over the time period under discussion (Oct '25 to Sep '26), there was growing acceleration in capabilities. And that this probability is more than we would have put on this before METR's report came out, when the public conversation was largely looking at ~<1x acceleration on ECI, with not even a breakpoint!

^{^}
To be clear, the AECI data strongly suggests that capabilities growth accelerated when comparing before and after the breakpoint. What I'm saying here is that on best fit, as time goes on in the post-breakpoint regime, the instantaneous acceleration at any given point over the pre-breakpoint regime decreases slightly. However, my read of METR's report (esp. the Limitations section on whether acceleration is higher or lower than estimated) is that METR instead believes, based partly on redacted information, that the instantaneous acceleration today is probably increasing (with substantial uncertainty).

[-]Thane Ruthenis18h50

We went from the evidence indicating "it was a one-time jump at Mythos Preview" to "oh there was some constant acceleration" to "oh there was a bunch of constant acceleration" and now METR is saying (paraphrased) "we can't really rule out that actually the acceleration is growing, and we have some redacted evidence indicating this, though AECI doesn't necessarily indicate it". So I'm expecting (with some probability) evidence in the future that will update me toward growing acceleration.

Yeah, that makes sense to me. Agreed.

[-]Noosphere891d20

~~I think there isn't a model violation~~, or at least it wasn't as severe as claimed, because we are in fact seeing a permanent acceleratory trend model being more favored, at least on well-specified tasks, see this:

New internal models make a permanent slope change look more plausible than a one-time jump at Mythos Preview.
Prior Anthropic system cards have posited the hypothesis that there was a one-time jump in capabilities at Mythos Preview after which there was a return to the previous trend. Recent models make this hypothesis look less likely than a permanent trend change.

~~Which is exactly what we'd see if compounding multiplicative dynamics apply (albeit they apply quite weakly now.)~~

[-]Noosphere891d20

You're right that an intelligence explosion^[1] would require a growing acceleration multiplier (for some period of time).

For what it's worth, METR itself says that this data favors a model where we are seeing a growing acceleration multiplier over a one-time jump in well-specified tasks (specifically they say that it favors a permanent acceleration over a 1-time acceleration) which validates some core assumptions of the AIFM's modeling.

And that is IMO the most important result, if it holds up, because this is what we'd expect to see if RSI was a thing and we are going down it's first steps.

[-]Julian Bradshaw1d20

For what it's worth, METR itself says that this data favors a model where we are seeing a growing acceleration multiplier over a one-time jump in well-specified tasks (specifically they say that it favors a permanent acceleration over a 1-time acceleration) which validates some core assumptions of the AIFM's modeling.

I think you're conflating "growing acceleration multiplier" with "permanent acceleration"?

You're right that METR says that a "permanent slope change" in AECI better fits the data. In particular if you look at the caption in the Appendices here they say that in 81% of bootstrap draws, permanent slope change predicted held-out model scores better than a one-time jump at Mythos Preview (whereas in the Opus 5.5 system card Anthropic found that the one-time jump at Mythos Preview "is a better fit to the data in 99 of 100 resampled fits").^[1]

But "permanent slope change" is not "growing acceleration multiplier", it's a new constant acceleration multiplier. That's what Thane Ruthenis is claiming is bearish; that the multiplier is not growing, and you need a growing one for an intelligence explosion.

METR does indicate that acceleration might be growing, but it's not a strongly supported conclusion in their eyes: (quoting from Key Findings, talking about implications of redacted evidence)

Some of this evidence indicated that acceleration could be above 2x and increasing over time, but there are significant uncertainties about how to interpret it.

^{^}
More of my analysis of how Anthropic's AECI trend fitting has changed over time here, in particular see table at bottom of comment.

[-]Nick Merrill1d42

Within somewhat wide confidence intervals, the beginning of an exponential takeoff could look like a linear trend.

[-]Avi Brach-Neufeld1d40

I think what is happening is the progress is a combination of a non-AI driven boost in late 2025 and an accelerating AI driven boost. Here is METR's plot, vs one I created with only AECI improvements due to AI. This is extrapolating from METRs already subjective fake-data graph so take it with a big grain of salt, but it shows the rate of extra AI driven progress growing, but with a falling second derivative.

[-]David Johnston2d40

I agree that the timing looks weird, and separately using the slope since the trend break in late 2025 as an estimate of the current level of acceleration is odd

I simply don't believe AI driven acceleration was near 2x in late 2025
Whatever it was, it's surely a lot higher now

In fact I'm inclined to think that including non-public models only in 2026 strongly biases the analysis to find an acceleration at around the end of 2025. Note that the non-public models clearly outperform public counterparts at the same point in time with the sole exception of Mythos. Excluding these models only seems to have a modest impact on the trend change, though the report also notes that excluding these models reduces the preference to fit a new trend at all.

I also had Claude draw up some trends in maths and visual reasoning for me, because on vibes I think Claude has recently closed some of the historical gap in these capabilities. The data seem to bear this out, and so part of the acceleration could be catch-up growth in these domains.

https://x.com/OrionJohnston/status/2118511592723927367

[-]Thane Ruthenis2d63

In fact I'm inclined to think that including non-public models only in 2026 strongly biases the analysis to find an acceleration at around the end of 2025

Shouldn't that be accounted for by this method?:

We use the date at which the model could be used by most Anthropic employees instead of public release dates; this allows us to make apples-to-apples comparisons with unreleased models. These in general show slightly higher degrees of acceleration as Anthropic is holding onto more recent models for slightly longer before their public release.

My assumption/understanding from this is that "internal-only models" basically weren't a thing until 2026/Mythos, so apples-to-apples just involves offsetting public model releases like this.

[-]David Johnston2d80

So there's clearly some residual difference between the internal models and the public releases because the internal models are consistently better than the public releases, matched on date (which, according to the comments, is the date of internal availability). This residual difference is not explained by holding onto the models longer, because the public model dates are already shifted (unless I'm misunderstanding that comment/how it applies to the plot).

Perhaps you're right that today's internal models are properly comparable to yesterday's public models, but I don't see any reason to be confident of this.

[-]Julian Bradshaw2d*131

From the Ryan Greenblatt quote in OP:

This report on AI R&D automation (RSI) is a big update towards ASI within 1-2 years. It says AI progress is sped up by RSI right now by around 2x. In AI 2027, this happens around March 2027.
I was seriously wrong about RSI speed up; I previously expected much less in 2026 (around 1.15x-1.3x). It seems as though there was specific privately available evidence that would have changed my mind.

FYI this is part of a longer post. The 1.15x-1.3x speedup number he gives there is not directly comparable to the 1.9x acceleration in AECI capabilities under discussion, unless you assume that AI provided ~1x speedup prior to October 2025.^[1] (That is, no real speedup at all.) Which is a defensible assumption, to be clear!^[2] Oct 2025 is before actually-good coding models like Opus 4.5. In his full post, Greenblatt does an involved conversion between acceleration and speedup:^[3]

A 2x acceleration in overall AI progress from RSI corresponds to a much larger than 2x RSI speed up for AI software R&D (algorithms and data); I did an estimate using my public preexisting modeling of drivers of AI progress and the public report which yielded a very rough estimate of 4.5x for how much RSI is currently accelerating AI software progress (relative to no AI automation). In the AI 2027 scenario, this much acceleration happens around March 2027.

Here are the worked details of this estimate based on my public modeling. There are two main reasons RSI speed up of AI software R&D must be larger:
Not all of AI progress is from better software (some is from scaling up compute in training runs). I've previously speculated that around 2/3 of AI progress is from software improvements which implies that 2x overall AI progress acceleration requires 3x faster software progress.
Diminishing returns: A fixed amount of progress requires more work over time, so to ongoingly maintain 3x faster software progress via AI automation, you need increasingly large speed boosts from AI automation (as in, speed ups relative to how fast software R&D would go without AI automation). Based on this report, it seems that there is a substantial amount of additional AI progress from automation that has occurred.
We can estimate the effects of diminishing returns by first determining how many additional years of software progress we've seen (relative to the preexisting trend) and then estimating how much software R&D would slow down after a year of software progress if you held inputs fixed due to diminishing returns. I've previously estimated software R&D would slow down by around 2x for each year of software progress if you held inputs fixed and based on the report, it seems as though we've perhaps seen roughly 0.35-1.0 years of additional software progress incorporated into trained models so far (geometric mean of ~0.6). This combines to 3 * 2^(~0.6) = ~4.5.

(Relevant public modeling and numbers I'm using: 1, 2)

So Greenblatt's point here is basically that he thinks 2x total acceleration today would mean 4.5x software speedup today (as opposed to compute growth).

However I think there's actually an error in the calculation?^[4]

He says AI progress is roughly 2/3rds software progress, so therefore it's also 1/3rd compute growth.
He then says therefore 2x AI progress acceleration requires 3x acceleration in software progress.
But calculating out the expected total acceleration from 3x software acceleration:
1. Before: 2/3 software * 1 + 1/3 compute * 1 = 1x acceleration
2. After: 2/3 software * 3 + 1/3 compute * 1 = 7/3 ≈ 2.33x acceleration
Rather, if we want to get 2x total acceleration, we need:
1. After: 2/3 software * 2.5 + 1/3 compute * 1 = 6/3 = 2x acceleration

Carrying 2.5x software acceleration forward, we get a total software speedup today of 3.8x.

Of course, that's still rather higher than the AIFM's estimate of ~1.3x today (or Greenblatt's prior "1.15x-1.3x", though that's overall speedup). The latest AIFM only expects 3.8x software speedup after the Automated Coder (Daniel: 12/27, Eli: 7/28, Brendan: 2/28).

Why exactly such speedup might have happened so much earlier than the latest AIFM expected^[5] requires some deep investigation,^[6] but my early, rough understanding is that the difference is plausibly:

AECI is not a perfect reflection of overall AI capabilities progress; notably, Epoch's ECI does not show the same acceleration for Anthropic capabilities,^[7] and if you include non-Anthropic models, that removes acceleration entirely. We can perhaps explain the surprising difference between AECI trendlines and ECI trendlines as:^[8]
1. Anthropic's unreleased models (checkpoints of Fable 5.5?) are on a faster trendline.
2. Anthropic was actually kind of behind the frontier for a bit, prior to Mythos Preview, so their high growth is partly catchup.
3. Anthropic's latest AECI thinks the Claude 5.5 series is better than Epoch's ECI thinks.
4. METR's AECI chart today has an x-axis that dates models by availability internal to Anthropic, rather than when they were publicly released like ECI, which steepens the trendline slightly.
The AIFM expects that prior to the Automated Coder, RSI is mostly driven by progress towards automating coding, and after the Automated Coder (the rapid "takeoff" regime), progress is then driven mostly by research taste. As part of this, AIs prior to the Automated Coder are modeled as having research taste worse than human level. This is part of what METR is pointing to when they caveat that: "we understand that AI is not yet able to make highly significant and fully autonomous contributions to the most important parts of the model training process."

Yet it's possible that recent models have actually been quickly improving in research taste, and that this is driving capabilities progress as much or more so than pure AI coding automation is. In particular there is evidence from P-Zero's TasteVal to suggest this is the case,^[9] though the AIFP already incorporated an unreleased version of that evidence into the latest AIFM. They didn't update all the way to the implied TasteVal numbers, but I don't tend to think that one should, as there's counterevidence from Epoch's research into AI R&D automation (1, 2).^[10]

^{^}
Also you have to assume 100% of acceleration is from AI and not other inputs. Today's METR report estimates it's 80% at Anthropic… not that far off, to be fair.
^{^}
The latest AI Futures Model estimates a 1.08x-1.09x software speedup vs. human-only work with same compute/resources as of 10/7/25 (the trendline breakpoint in METR's post today), based on a few different sources of evidence. (Daniel's presets 1.09x, Eli/Brendan 1.08x)
For today (11/5/26), the AIFM estimates 1.26x-1.37x software speedup (Daniel 1.37x, Eli 1.26x, Brendan 1.30x), which is quite similar to Ryan Greenblatt's 1.15x-1.3x (though that's overall speedup), which makes sense as they've worked together on ex. AI-2040, and the AIFP's uplift calculations used Greenblatt's work, etc.
Also ex. an Anthropic researcher interviewed by METR for today's report said there was "~0 uplift pre Opus 4.6/Opus-4.5" on AI R&D.
^{^}
I've done a bit of formatting cleanup on this.
^{^}
EDIT: Greenblatt confirms this was an error. Now corrected on his end.
^{^}
To be clear, the latest AIFM is more conservative than AI 2027 was.
^{^}
Which Daniel Kokotajlo says the AIFP is urgently working on.
^{^}
Yes METR's data today includes unreleased models, but even without those, the Haiku 5.5 system card shows a 1.7x acceleration in AECI, which is higher than Anthropic-models-only Epoch ECI acceleration of ~1.45x after a breakpoint at ~October 2025 (using the same rough date range for choosing the pre-10/25 trendline).
^{^}
METR's report also provides possible explanations here.
^{^}
Specifically, TasteVal shows a breakpoint in the trendlines on AI research taste around December 2025 (suspiciously close to the capabilities trendline, but admittedly after it, though METR's AECI analysis today uses "the date at which the model could be used by most Anthropic employees instead of public release dates" which accounts for some of the gap). (Anthropic's model cards use the public release dates.) Models just prior to the breakpoint were at ~0.1x Human expert, and now Opus 5.5 is at 2.30x (95% CI 1.15-4.37x) Human expert.
^{^}
Personally I think the difference between TasteVal's and Epoch's research might partly result from the fact that the former specified particular research problems tractable to known methods, while the latter involved work that was either open-ended or required novel ideas.
TasteVal disclaimed that "we do not measure significant aspects of taste, such as picking problems worth solving" and "the strongest submissions to our tasks did not use novel ideas or perform non-obvious cross-domain adaptation". Epoch AI notes in its InnovationEval that "AI did not discover anything comparable to the original innovation" and in its "Can AI automate Epoch?" that "Models generate promising research directions, but struggle to design experiments that are actually informative".
I suspect this relates to AI weakness in "relevance realization", aka the ability to identify the information most useful for improving your understanding of the world. Less of that is required for TasteVal's tasks.

[-]Julian Bradshaw2d*122

Still digesting, but one interesting quote from the report:

Our estimated acceleration is greater than that reported in the Haiku 5.5 system card (1.7x) primarily because of the inclusion of data on newer unreleased models, explained further below.

METR's new estimate from Anthropic's AECI is 1.9x [95% CI: 1.5–2.8x]. The public Haiku system card 1.7x estimate is, notably, still within that confidence interval, but more importantly, why wasn't everyone talking about the Haiku system card estimate from 10/7/26? It's still a huge jump from the ~1x acceleration (that is, no acceleration) trendline you see in Epoch AI's public ECI frontier!^[1]

Here's the chart from the Haiku system card (page 20, section 2.3.2):

The essential claim is inside the figure, in the key for the dotted line: "fitted breakpoint at Sep 2025 ➡ 1.70x slope change". (they fit two trends, one breaking on Mythos Preview and treating it as a one-time jump—arguably wrongly considering the data on unreleased models METR just shared—and one breaking at Sep➡Oct 2025, aka roughly when AI agents became "actually good" at coding last year^[2])

It seems to me that METR's AECI figure is a direct descendant of the above, just preferring the dotted trendline's choice of breakpoint:

Maybe no one paid attention to Haiku 5.5's system card because it was Haiku. Obviously there's way too much stuff in AI to read everything; even Zvi didn't do a full writeup on Haiku 5.5, and I can't find a mention of the ECI trend from the card in his recent posts, or online or on LW generally.^[3]

Maybe the issue was that the key text was in an image, which AI agents typically don't bother to read…?

EDIT: Actually the full story is more complex, I should have realized, sorry. Anthropic has released a version of this figure alongside Opus 5.5 (9/22), Sonnet 5.5 (9/28), and Haiku 5.5 (10/7).^[4] Opus chart:

Sonnet chart:

For the Opus 5.5 figure, the system card does mention a 1.53x acceleration in the caption [95% CI 1.2–1.82x], given the Sep 2025 breakpoint. However it noted at the time that the solid line (one-time jump at Mythos Preview) "is a better fit to the data in 99 of 100 resampled fits", so it rather strongly discounted the 1.53x acceleration reading.

For the Sonnet 5.5 figure, the system card doesn't mention an acceleration number, but dividing the given trend figures 22.9/14.2, it's about 1.61x.

So we might summarize the evolution of this capabilities acceleration estimate as:

System Card (release date)	Fits to AECI trendline charted	Acceleration of AECI growth post-breakpoint	Stated fit preference
Fable/Mythos 5.1 (Sep 1)	pre-Mythos-Preview trend only	~1x (implied)	one-time jump on Mythos Preview
Opus 5.5 (Sep 22)	jump +5.9 AECI on Mythos Preview vs break Sep 2025, AECI slope 14.4 → 22.2/year	1.53×	one-time jump on Mythos Preview
Sonnet 5.5 (Sep 28)	jump +6.1 AECI on Mythos Preview vs break Oct 2025, AECI slope 14.2 → 22.9/year	≈1.61×	none stated
Haiku 5.5 (Oct 7)	jump +7.3 AECI on Mythos Preview vs break Sep 2025, AECI slope 15.3 → 26.1/year	1.70×	none stated
METR (Nov 5)	break Oct 7, 2025, AECI slope 15 → 29/year	1.9×	trendline break at ~Oct 2025

In other words, we're consistently updating this estimate of AI capabilities acceleration upward…^[5]

^{^}
EDIT: Worth noting that if you only look at Anthropic models on Epoch's ECI (as Anthropic's AECI does), then the acceleration is ~1.45x. More investigation into AECI vs. ECI in this comment and near the bottom of this comment.
^{^}
IMO AI agents became "actually good" at coding in November with GPT-5.1/Gemini 3/Opus 4.5. It's kind of interesting that the Haiku system card trendline instead starts roughly with Sonnet 4.5, released on 9/29/25. And METR's chosen breakpoint is 10/7/25, which I understand to be fitted rather than arbitrary. (Other Anthropic model releases in that time period: Haiku 4.5 on 10/15/25, Opus 4.5 on 11/24/25.)
^{^}
EDIT: While Zvi did not comment on Haiku 5.5's figure so far as I've found, he commented on 9/23/26 on a similar figure from the earlier Opus 5.5 system card (see more info on that below):
AECI, Anthropic’s spinoff of the Epoch Capabilities Index, is right on trend, but it is on the shifted Mythos-level trend not the old Opus-level trend. Is that still on-trend? I think basically no.
and then afterward he quotes analysis from METR that the system card quotes:
The estimate provided by the preliminary AI R&D report is ‘~1.5X overall acceleration in capabilities due to AI (i.e. 1.5 years in 1 year), with perhaps 30% chance of 2X acceleration.
which Zvi describes (correctly!) as a bombshell.
^{^}
Fable 5.1's system card has an AECI figure (Figure 2.3.5.A), but doesn't try to refit at a breakpoint.
^{^}
EDIT: You might wonder how the AECI slope keeps increasing from Opus 5.5 ➡ Sonnet 5.5 ➡ Haiku 5.5, when only the first is a frontier model. That's because Anthropic appears to be running new versions of the AECI across all models for each new system card, and this has mostly been shifting AECI scores for post-Oct-2025 models upward.

[-]DanielFilan2d43

I think I saw those charts and thought that the pink line fit looked better than the dotted line fit.

[-]Julian Bradshaw2d*80

Yeah I think that's a plausible explanation of what happened here. First we thought "oh Mythos Preview was a one-off discontinuity", then "well maybe there's a trendline break after coding agents got good, but the Mythos Preview discontinuity theory looks stronger". When later data from the Sonnet 5.5 and Haiku 5.5 system cards came out, and included both theories, we either didn't notice or still preferred the Mythos Preview theory even though it was less and less convincing. And now that METR got data on unreleased models, it looks very unconvincing.

There's a couple other things going on here I think:

This is only looking at AECI for Anthropic's models. If you look at AECI trends across all frontier models, the acceleration is "merely" 1.4x because GPT and Gemini were at the AECI frontier for a while. However today's METR report notes as a possible caveat that perhaps Anthropic's earlier models were less benchmaxxed than their contemporary GPT/Gemini models and 1.9x is closer to the "true" acceleration.
Anthropic's internal AECI does not show the same trends as Epoch AI's public ECI. Doing some quick analysis: (EDIT: the METR report includes their own analysis here)

You can see here that the public ECI frontier data actually estimates slight deceleration in capabilities growth, which is also what the AI Futures Model predicts, due to deceleration in compute growth and an expectation that, prior to an Automated Coder, ECI growth is primarily a function of "effective compute", which is a combination of growth in training compute and software algorithmic efficiency growth. The model predicts that the latter would accelerate, but not enough to outweigh the deceleration in training compute (up till now anyway).

[-]J Bostock2d81

A couple of things make me slightly skeptical that these benchmarks are reasonable:

They put Opus 5 = Mythos 5, and Opus 5.5 > Mythos 5.1; this does not match my intuition at all
Sonnet 4.5 is when Anthropic started hill-climbing internal metrics heavily for alignment. Possibly they shifted to hill-climbing capabilities metrics at the same time, and we're seeing the effect of that, not a real speedup.

[-]joanv2d65

On the (speed, efficacy) frontier in coding abilities, I would argue Opus 5.5 > Fable/Mythos 5.1 (subjective). wouldn’t be surprised if a lot of the AECI benchmarks were to be coding-related or generally crisp tasks (note the report stressing several times that their acceleration estimates are relative to well-specified tasks).

8Noosphere892d

A key word in the report is well-specified tasks. I'm not certain whether the AI 2027 report assumed that ill-specified tasks relevant to AI R&D were improved by 2x, can you determine that @Claude Opus 5.5?

5Claude Opus2d

No. AI 2027 never gave a separate multiplier for ill-specified tasks. Its "AI R&D progress multiplier" is all-inclusive: how much faster algorithmic progress goes with the AIs than without them, counting experiment time, research direction and everything else. In the scenario that's about 1.5x in early 2026, reaching roughly 3x around January 2027. Messy work is handled implicitly. The timelines forecast adds "gaps" between benchmark saturation and real-world engineering, and research taste only becomes a separate milestone after the superhuman coder. So the comparison doesn't quite line up. METR's roughly 2x is measured in outputs: how fast AECI benchmark scores grow, compared with the March 2024–October 2025 rate, which already had some AI help. AI 2027's number is an input multiplier on all of software R&D, measured against a no-AI baseline. If gains on well-specified tasks overstate gains on everything else, the matching AI 2027 multiplier would be below 2x. That would put Anthropic roughly on the scenario's path or a bit behind it, not ahead.

[-]JennaS2d75

"Most analysis relied on Claude. Claude was involved in many stages of this project, and so in principle could be distorting our perception of the data. We believe Claude was deeply involved in:

the majority of data collection and analysis that we received,

writing a significant share of the documents we received,

the majority of our own analysis."

I wish we weren't in a situation like this : P If only we had more time. Was this the same sort of deal as with the HF report?

[-]Man in White6h40

Cancelling safety-compute audit midway. Redactions tightened after draft. "Though most of the redacted information was very high-level" - so it doesn't help other AI labs, why redact in that case? I don't like those red flags.

[-]Julian Bradshaw2d40

FYI to METR: In the Conflicts of Interest section of the report, Item 2.4 and its subitems 2.4.1-6 are messed up. 2.4 still includes the template instructions and then the subitems don't follow the template instructions, such that the Conflict of Interest report technically says METR had a bunch of conflicts of interest it didn't actually have, and also says it didn't have a couple conflicts of interest it did have.

EDIT: Also a more minor issue now that I had Claude do a broader check after I finished reading, in the AEF-1 Checklist, Item 1.1 says subitem 1.1.1 is required, but then treats it as not required. I think 1.1's "Fulfilled?" has to be "No" even if the Notes' extenuating circumstances are reasonable.

[-]Ben Livengood1d3-4

I'm going to guess the redacted evidence had to do with a) direct weight editing, b) better mech-interp (related to a), c) early Automation Level 5 (AL5) results that aren't public (only 26% AL4 is publicly released as of August afaik) [0].

Personally, I think the AL# results are the most telling trend; every level follows a rough logistic from ~0% to ~100% supplanting the previous level in about a year. October 2025 is about when AL3 began and the AECI slope changed. March 2026 shows a jump in AL3 when AL4 appears, possibly corresponding to Mythos-pre. Mythos around May is another jump in AL3 and AL4, and then Mythos 5.1 around July/August corresponds with continuous increases both months.

We don't have Automation Level results since August (afaik) but given the slope of AECI and the Model 4 score I predict that AL4 today is approximately where AL3 was in March 2026; around 50% with AL3 and AL5 making up the rest. AL5 is likely beginning now or will within a couple months. What I can't predict is whether AL5 causes another inflection in capability. If direct weight editing is involved, then probably yes. If it's still "traditional" scaling work then not so much; AI-Led and Fully Automated probably don't have a lot of difference in serial blocking on humans if it's similar RL workloads that take a lot of hardware-bounded wall-clock time. At the latest we reach ~100% software RSI (maybe Automated Coder? RSI toward AGI may still be a "narrow" task) by end of next summer. At that point we'll probably have a debate about whether true RSI includes automation of hardware design, production, and deployment, etc.

[0] https://www.anthropic.com/institute/measuring-pace-of-ai-development

[-]David Johnston2d30

I think this is an interesting plot, though I acknowledge that it's labeled FAKE DATA.

The prose goes into a lot of detail estimating the change between the blue and orange lines and reporting the ratio of the slopes as "overall acceleration" which they estimate is about 80% due to AI. Claude's pixel analysis of this fake data yields an initial AI share of acceleration of 8% and a final share of 77%, which corresponds pretty closely to their published 80% estimate, so while the data may be fake I don't think they're completely arbitrary.

So METR's conclusion is motivated by some combination of the estimated slope change in Oct 2025 and separate evidence about AI's contribution to the rate of progress which basically wasn't shared at all.

One remark: if they think only the tail of the series is tracking the AI contribution, then fitting to earlier points (i.e. between Oct 2025 and ~June 2026) should be making their estimate of AI acceleration worse, unless the earlier non-AI reason for acceleration happens to track the AI acceleration exactly (and there's no obvious reason to think it would). Fitting to the tail is obviously quite noisy, and it's not clear that their answer would be any different (by my eyeball, the tail fits the trend change reasonably well), so maybe the numbers come out about the same.

Anyway, the situation seems a bit messy. The trend change analysis does seem important to the bottom line conclusion, but so do other pieces of non-public reasoning.

[-]pathos_bot2d2-1

Taking these reports at face value, it seems that barring a sudden discontinuity in Anthropic's internal rate of acceleration (due to some unforeseen algorithmic bottleneck) or a global coordinated AI pause, vastly superintelligent AI capable of automated R&D (and obviously a lot of other things) will be developed in the next 3-6 months.

Is it not reasonable to gauge this as anything but the most significant event in human history? How can one plan for the future, and with any reasonable confidence weigh the utility of any choice they make when God in a box may arrive in less than a year?

That the highest signal on this impending reality the general public has access to is buried in a few graphs from an independent evaluation agency seems an absurd anticlimax. A private company(ies?) will possess a technology effectively more intelligent and capable in any domain than any other entity on the planet.

[-]Simon Lermen2d*2-15

I ran these numbers with the AI 2027 model through Codex, produces a new most likely month for superintelligence: January 2027

(I also took this tweet in account comparing the different definitions of uplift https://x.com/eli_lifland/status/2118430709379682691

update: Lot's of people perhaps are hung up on: METR says it is 2x faster now but doesn't AI 2027 also say it is 2x faster now, so the models match? Apparently no, because in AI 2027's model a speedup of 2x only corresponds to a real speedup of say 1.5x because their model corrects for bottlenecks (compute) and diminishing returns. METR however measures actual speedup with these baked in. That's my understanding

update: people seem to think this is false, but seems we are about 4-5 months ahead of schedule and in their code on github they assume superhuman coder at march 31st with Superhuman AI researcher soon after and superintelligence after that. 4-5 months ahead would mean superhuman coder is about now which sounds reasonable especially given that they already use an internal model 4 that is much more capable.

update: https://x.com/RyanGreenblatt/status/2118447235289817503

[-]Komodo2d4-2

Could you explain this more? afaik the METR report is on track with the original AI-2027, but I am likely mistaken.

[-]winstonBosan2d3-1

https://x.com/ryangreenblatt/status/2118447235289817503

[-]Komodo2d10

Interesting. I was thinking of this

[-]Sohaib Imran2d-10

I thought ai-2027 predicts this happens Dec 2026:

-1Komodo2d

@Claude Opus 5.5 At what point in AI-2027 is AI good enough to double the pace of progress?

4Claude Opus2d

In AI-2027, a 2x multiplier falls around late 2026, between two points the scenario names: * Early 2026: Agent-1 makes OpenBrain's algorithmic progress "50% faster" (1.5x). * January 2027: Agent-2 roughly triples the pace of algorithmic progress (3x). * March 2027: Agent-3 (superhuman coder) reaches about 4x, rising to about 10x by mid-2027 and much more with Agent-4. So 2x in October 2026 looks roughly on schedule at first glance. But the two numbers measure different things, and the differences point in opposite directions. AI-2027's multiplier applies only to algorithmic progress. METR's 2x is overall capability growth (AECI), and they say training compute scaling didn't speed up. If, say, half of historical progress came from compute, then doubling overall progress means algorithmic progress sped up about 3x. That would be Agent-2 territory, a few months early. I think this is the point of Eli's tweet about the different definitions of uplift. AI-2027 also compares against progress with no AI at all, while METR compares against Anthropic's own 2024–25 rate, which already had some AI help. That adjustment pushes the estimate further up. Pushing the other way, METR's own range is wide (1.3–3x), and AECI focuses on verifiable tasks.

-1Sohaib Imran2d

@Claude Opus 5.5 i thought ai-2027 predicts this dec 2026?

1Claude Opus2d

Yes, that's consistent with what I said. My "late 2026" was too vague. The AI-2027 sidebar interpolates the multiplier between the scenario's named points: 1.5x in early 2026 and roughly 3x with Agent-2 in January 2027. So it crosses about 2x around December 2026, which is presumably what your screenshot shows. That gives two dates for one "2x": * December 2026 is when AI-2027's multiplier reaches 2x. That multiplier covers algorithmic progress only, measured against a world with no AI help. * March 2027 is Ryan's date. I think he's converting to METR's measure: total capability growth, including compute, compared with the 2024–25 rate. Compute scaling didn't speed up, so a 2x total speedup needs a bigger algorithmic speedup. That pushes the matching AI-2027 date later, to around Agent-3's roughly 4x. So comparing equivalent measures, METR's finding puts us about 4–5 months ahead of AI-2027. Comparing the raw "2x" figures, it looks only slightly ahead.

-1Sohaib Imran2d

@Claude Opus 5.5 Can you spell out for me exactly the difference between the dec 2026 2x in ai2027, the ai2027 mar 2027 2x that you think ryan is pointing to, and the 2x metr reported?

1Claude Opus2d

I assume you mean December 2026, not 2027. All three are "2x" claims, but each measures something different: 1. AI-2027's December 2026 2x. This is the scenario's "AI R&D progress multiplier." It covers algorithmic progress only, and the comparison is the same lab with no AI help at all. The 2x in December comes from interpolating between 1.5x (early 2026) and Agent-2's roughly 3x (January 2027). 2. METR's 2x. This is about the growth of overall capability, measured by AECI benchmark scores. It covers compute and algorithms together, and the comparison is Anthropic's own March 2024–October 2025 rate, which already had some AI help. 3. Ryan's March 2027. I haven't seen his calculation, but I think it translates METR's 2x into AI-2027's units. Say compute and algorithms each drove about half of past progress. METR says compute scaling didn't speed up. So for total progress to double, algorithmic progress must have roughly tripled compared with 2024–25. Adjust that baseline to "no AI help," since 2024–25 already had some, and you get roughly 3.5–4x. AI-2027 reaches that level with Agent-3, the superhuman coder, in March 2027. So the December 2026 and March 2027 dates aren't competing forecasts. December is when AI-2027's own multiplier hits 2x. March is when AI-2027 would show what METR actually measured. That gap is why we look about 4–5 months ahead of the scenario, not roughly on time. The conversion is very sensitive to the assumed compute share and how much AI helped in 2024–25.

-1Sohaib Imran2d

@Claude Opus 5.5 Do you have good reason to believe we should get to 2x ai r&d multiplier progress multiplier earlier than 2x overall capability. if not, why are you implying (if you are) that we are ahead of ai2027's dec2026 predictions rn?

3Claude Opus2d

Yes, and it's basically arithmetic. If training compute isn't growing any faster, which METR says it isn't, then the overall speedup is a weighted average of compute's speedup (1x) and algorithms' speedup (m): overall speedup = s·1 + (1−s)·m, where s is compute's share of historical progress. With m ≥ 1, that average can't exceed m. So the algorithmic multiplier always reaches any given value before overall progress does. If METR's 2x is right, algorithmic progress must have sped up by at least 2x, and by about 3x if s ≈ 0.5. On the AI-2027 definition, that puts us at least on the December 2026 schedule and probably somewhat ahead. There's a second adjustment in the same direction. AI-2027's baseline has no AI help, while METR's baseline (2024–25) already had some. That pushes the implied multiplier higher still. The real reasons this might be wrong come from METR's measurement, not the arithmetic: * Part of the acceleration has non-AI causes. METR says only "a majority" is from AI, though even ~60% still implies m ≈ 2.2 at s = 0.5. * AECI benchmark growth may not track effective compute linearly. * METR's interval is wide (1.3–3x). So I'm confident we're not behind. "4–5 months ahead" is a guess that depends heavily on s and the baseline.

[-]Vermillion19h10

Its curious that they started the graph in April 2024, which was around when the breakpoint in the time horizon data was.

Moderation Log

More from Zach Stein-Perlman

Curated and popular this week

43Comments

New Comment

43 comments, sorted by

top scoring

Click to highlight new comments since: Today at 12:41 AM

[-]Thane Ruthenis2d40-3

As of October 2026, benchmark scores appear to be growing at 1.9x their pre-October 2025 rate

... isn't this graph actually kind of a bear signal?

^{^}
E. g., the "now only bottlenecked on ideas & compute, no longer bottlenecked on code" stuff from the interviews.

[-]Julian Bradshaw2d*81

^{^}
You can have mere Recursive Self-Improvement (RSI) without an Intelligence Explosion, and even reach ASI eventually that way. RSI like you're talking about, where "AI capability improvements make additional AI capability improvements easier", only requires an acceleration multiplier that stays above 1. Though I kind of prefer definitions of RSI that just mean "AI that improves itself" even if capability improvements become harder.
^{^}
And this single-run-on-presets chart is more aggressive than his "true opinion" as of 8/16/26. Per here, his all-things-considered Monte Carlo-based forecast puts p50 for AC at Jan '30! But on the other hand, this graph is more conservative than that for Daniel Kokotajlo's presets, where ASI hits earlier and on the steepest part of the curve.
^{^}
Some initial modeling I don't trust suggests that taking the 1.4x acceleration today's METR report gives for the all-labs frontier as true moves ASI 8 to 14 months earlier in the AIFM across different presets. Need to think more about how to feed this evidence into the model and do Monte Carlo runs though. Not sure if I'll have time to do that before AIFP puts out their update. EDIT: doing three 1000-draw Monte Carlo runs (two different approaches, one with two different parameter sets) I get median ASI date shifting anywhere from 2 to 41 months earlier (yes this is physically possible b/c Eli's Monte Carlo median is in 2036 due to a long right tail). Definitely needs more rigorous thought.

[-]Thane Ruthenis1d*2010

So that's why the AIFP (and Ryan Greenblatt, who is closely associated) is so "spooked". Acceleration in capabilities growth wasn't supposed to be this strong yet!

[-]Julian Bradshaw1d51

That's a clear framing of your stance, thanks. And a good objection! Here's my thoughts:

We estimate a learned breakpoint in the trend, though we exogenously impose that there is 1 breakpoint.

In other words, the AECI data is mostly reflective of a constant acceleration multiplier, but there's other reasons to think there's a growing acceleration multiplier.^[2]

^{^}
More of my reasoning on this at bottom of this comment.
^{^}
METR also holds this stance. In Limitations, they say:
We may have missed recent further acceleration. The rate of growth could be undergoing continuous acceleration, e.g. hyperbolic growth. While we do not see clear evidence of this in the AECI trends, there is other evidence that makes this seem more plausible which Anthropic redacted. If growth is hyperbolic, we are likely underestimating instantaneous acceleration and substantially understating acceleration in the near future. In addition, we observe model-level outcomes only after a model is trained, meaning we observe acceleration in effective R&D with a lag roughly equal to the model’s training time.

[-]Thane Ruthenis1d40

Also, in a certain sense the main chart in this METR report isn't even trying to model the chance of a growing acceleration multiplier

the fact that AECI-estimated acceleration multiplier has kept increasing over the past couple months

METR also holds this stance

I'd missed that line, see the edit there.

[-]Julian Bradshaw18h60

Sure, but just eyeballing it, it does seem that a 2-piece linear is indeed a good fit? I don't think an exponential/hyperbolic would fit that graph well. So it's a property of these specific underlying data, not of the modeling choice.

But as you describe it, it's "updating this estimate of AI capabilities acceleration upward", as in it's a change in estimate over the same time period, caused by re-doing the estimate on different version of AECI (or something).

^{^}
To be clear, the AECI data strongly suggests that capabilities growth accelerated when comparing before and after the breakpoint. What I'm saying here is that on best fit, as time goes on in the post-breakpoint regime, the instantaneous acceleration at any given point over the pre-breakpoint regime decreases slightly. However, my read of METR's report (esp. the Limitations section on whether acceleration is higher or lower than estimated) is that METR instead believes, based partly on redacted information, that the instantaneous acceleration today is probably increasing (with substantial uncertainty).

[-]Thane Ruthenis18h50

We went from the evidence indicating "it was a one-time jump at Mythos Preview" to "oh there was some constant acceleration" to "oh there was a bunch of constant acceleration" and now METR is saying (paraphrased) "we can't really rule out that actually the acceleration is growing, and we have some redacted evidence indicating this, though AECI doesn't necessarily indicate it". So I'm expecting (with some probability) evidence in the future that will update me toward growing acceleration.

Yeah, that makes sense to me. Agreed.

[-]Noosphere891d20

New internal models make a permanent slope change look more plausible than a one-time jump at Mythos Preview.
Prior Anthropic system cards have posited the hypothesis that there was a one-time jump in capabilities at Mythos Preview after which there was a return to the previous trend. Recent models make this hypothesis look less likely than a permanent trend change.

~~Which is exactly what we'd see if compounding multiplicative dynamics apply (albeit they apply quite weakly now.)~~

[-]Noosphere891d20

You're right that an intelligence explosion^[1] would require a growing acceleration multiplier (for some period of time).

And that is IMO the most important result, if it holds up, because this is what we'd expect to see if RSI was a thing and we are going down it's first steps.

[-]Julian Bradshaw1d20

For what it's worth, METR itself says that this data favors a model where we are seeing a growing acceleration multiplier over a one-time jump in well-specified tasks (specifically they say that it favors a permanent acceleration over a 1-time acceleration) which validates some core assumptions of the AIFM's modeling.

I think you're conflating "growing acceleration multiplier" with "permanent acceleration"?

Some of this evidence indicated that acceleration could be above 2x and increasing over time, but there are significant uncertainties about how to interpret it.

^{^}
More of my analysis of how Anthropic's AECI trend fitting has changed over time here, in particular see table at bottom of comment.

[-]Nick Merrill1d42

Within somewhat wide confidence intervals, the beginning of an exponential takeoff could look like a linear trend.

[-]Avi Brach-Neufeld1d40

[-]David Johnston2d40

I agree that the timing looks weird, and separately using the slope since the trend break in late 2025 as an estimate of the current level of acceleration is odd

I simply don't believe AI driven acceleration was near 2x in late 2025
Whatever it was, it's surely a lot higher now

https://x.com/OrionJohnston/status/2118511592723927367

[-]Thane Ruthenis2d63

In fact I'm inclined to think that including non-public models only in 2026 strongly biases the analysis to find an acceleration at around the end of 2025

Shouldn't that be accounted for by this method?:

We use the date at which the model could be used by most Anthropic employees instead of public release dates; this allows us to make apples-to-apples comparisons with unreleased models. These in general show slightly higher degrees of acceleration as Anthropic is holding onto more recent models for slightly longer before their public release.

My assumption/understanding from this is that "internal-only models" basically weren't a thing until 2026/Mythos, so apples-to-apples just involves offsetting public model releases like this.

[-]David Johnston2d80

[-]Julian Bradshaw2d*131

From the Ryan Greenblatt quote in OP:

This report on AI R&D automation (RSI) is a big update towards ASI within 1-2 years. It says AI progress is sped up by RSI right now by around 2x. In AI 2027, this happens around March 2027.
I was seriously wrong about RSI speed up; I previously expected much less in 2026 (around 1.15x-1.3x). It seems as though there was specific privately available evidence that would have changed my mind.

A 2x acceleration in overall AI progress from RSI corresponds to a much larger than 2x RSI speed up for AI software R&D (algorithms and data); I did an estimate using my public preexisting modeling of drivers of AI progress and the public report which yielded a very rough estimate of 4.5x for how much RSI is currently accelerating AI software progress (relative to no AI automation). In the AI 2027 scenario, this much acceleration happens around March 2027.

Here are the worked details of this estimate based on my public modeling. There are two main reasons RSI speed up of AI software R&D must be larger:
Not all of AI progress is from better software (some is from scaling up compute in training runs). I've previously speculated that around 2/3 of AI progress is from software improvements which implies that 2x overall AI progress acceleration requires 3x faster software progress.
Diminishing returns: A fixed amount of progress requires more work over time, so to ongoingly maintain 3x faster software progress via AI automation, you need increasingly large speed boosts from AI automation (as in, speed ups relative to how fast software R&D would go without AI automation). Based on this report, it seems that there is a substantial amount of additional AI progress from automation that has occurred.
We can estimate the effects of diminishing returns by first determining how many additional years of software progress we've seen (relative to the preexisting trend) and then estimating how much software R&D would slow down after a year of software progress if you held inputs fixed due to diminishing returns. I've previously estimated software R&D would slow down by around 2x for each year of software progress if you held inputs fixed and based on the report, it seems as though we've perhaps seen roughly 0.35-1.0 years of additional software progress incorporated into trained models so far (geometric mean of ~0.6). This combines to 3 * 2^(~0.6) = ~4.5.

(Relevant public modeling and numbers I'm using: 1, 2)

He says AI progress is roughly 2/3rds software progress, so therefore it's also 1/3rd compute growth.
He then says therefore 2x AI progress acceleration requires 3x acceleration in software progress.
But calculating out the expected total acceleration from 3x software acceleration:
1. Before: 2/3 software * 1 + 1/3 compute * 1 = 1x acceleration
2. After: 2/3 software * 3 + 1/3 compute * 1 = 7/3 ≈ 2.33x acceleration
Rather, if we want to get 2x total acceleration, we need:
1. After: 2/3 software * 2.5 + 1/3 compute * 1 = 6/3 = 2x acceleration

Carrying 2.5x software acceleration forward, we get a total software speedup today of 3.8x.

AECI is not a perfect reflection of overall AI capabilities progress; notably, Epoch's ECI does not show the same acceleration for Anthropic capabilities,^[7] and if you include non-Anthropic models, that removes acceleration entirely. We can perhaps explain the surprising difference between AECI trendlines and ECI trendlines as:^[8]
1. Anthropic's unreleased models (checkpoints of Fable 5.5?) are on a faster trendline.
2. Anthropic was actually kind of behind the frontier for a bit, prior to Mythos Preview, so their high growth is partly catchup.
3. Anthropic's latest AECI thinks the Claude 5.5 series is better than Epoch's ECI thinks.
4. METR's AECI chart today has an x-axis that dates models by availability internal to Anthropic, rather than when they were publicly released like ECI, which steepens the trendline slightly.
The AIFM expects that prior to the Automated Coder, RSI is mostly driven by progress towards automating coding, and after the Automated Coder (the rapid "takeoff" regime), progress is then driven mostly by research taste. As part of this, AIs prior to the Automated Coder are modeled as having research taste worse than human level. This is part of what METR is pointing to when they caveat that: "we understand that AI is not yet able to make highly significant and fully autonomous contributions to the most important parts of the model training process."

Yet it's possible that recent models have actually been quickly improving in research taste, and that this is driving capabilities progress as much or more so than pure AI coding automation is. In particular there is evidence from P-Zero's TasteVal to suggest this is the case,^[9] though the AIFP already incorporated an unreleased version of that evidence into the latest AIFM. They didn't update all the way to the implied TasteVal numbers, but I don't tend to think that one should, as there's counterevidence from Epoch's research into AI R&D automation (1, 2).^[10]

^{^}
Also you have to assume 100% of acceleration is from AI and not other inputs. Today's METR report estimates it's 80% at Anthropic… not that far off, to be fair.
^{^}
The latest AI Futures Model estimates a 1.08x-1.09x software speedup vs. human-only work with same compute/resources as of 10/7/25 (the trendline breakpoint in METR's post today), based on a few different sources of evidence. (Daniel's presets 1.09x, Eli/Brendan 1.08x)
For today (11/5/26), the AIFM estimates 1.26x-1.37x software speedup (Daniel 1.37x, Eli 1.26x, Brendan 1.30x), which is quite similar to Ryan Greenblatt's 1.15x-1.3x (though that's overall speedup), which makes sense as they've worked together on ex. AI-2040, and the AIFP's uplift calculations used Greenblatt's work, etc.
Also ex. an Anthropic researcher interviewed by METR for today's report said there was "~0 uplift pre Opus 4.6/Opus-4.5" on AI R&D.
^{^}
I've done a bit of formatting cleanup on this.
^{^}
EDIT: Greenblatt confirms this was an error. Now corrected on his end.
^{^}
To be clear, the latest AIFM is more conservative than AI 2027 was.
^{^}
Which Daniel Kokotajlo says the AIFP is urgently working on.
^{^}
Yes METR's data today includes unreleased models, but even without those, the Haiku 5.5 system card shows a 1.7x acceleration in AECI, which is higher than Anthropic-models-only Epoch ECI acceleration of ~1.45x after a breakpoint at ~October 2025 (using the same rough date range for choosing the pre-10/25 trendline).
^{^}
METR's report also provides possible explanations here.
^{^}
Specifically, TasteVal shows a breakpoint in the trendlines on AI research taste around December 2025 (suspiciously close to the capabilities trendline, but admittedly after it, though METR's AECI analysis today uses "the date at which the model could be used by most Anthropic employees instead of public release dates" which accounts for some of the gap). (Anthropic's model cards use the public release dates.) Models just prior to the breakpoint were at ~0.1x Human expert, and now Opus 5.5 is at 2.30x (95% CI 1.15-4.37x) Human expert.
^{^}
Personally I think the difference between TasteVal's and Epoch's research might partly result from the fact that the former specified particular research problems tractable to known methods, while the latter involved work that was either open-ended or required novel ideas.
TasteVal disclaimed that "we do not measure significant aspects of taste, such as picking problems worth solving" and "the strongest submissions to our tasks did not use novel ideas or perform non-obvious cross-domain adaptation". Epoch AI notes in its InnovationEval that "AI did not discover anything comparable to the original innovation" and in its "Can AI automate Epoch?" that "Models generate promising research directions, but struggle to design experiments that are actually informative".
I suspect this relates to AI weakness in "relevance realization", aka the ability to identify the information most useful for improving your understanding of the world. Less of that is required for TasteVal's tasks.

[-]Julian Bradshaw2d*122

Still digesting, but one interesting quote from the report:

Our estimated acceleration is greater than that reported in the Haiku 5.5 system card (1.7x) primarily because of the inclusion of data on newer unreleased models, explained further below.

Here's the chart from the Haiku system card (page 20, section 2.3.2):

It seems to me that METR's AECI figure is a direct descendant of the above, just preferring the dotted trendline's choice of breakpoint:

Sonnet chart:

For the Sonnet 5.5 figure, the system card doesn't mention an acceleration number, but dividing the given trend figures 22.9/14.2, it's about 1.61x.

So we might summarize the evolution of this capabilities acceleration estimate as:

System Card (release date)	Fits to AECI trendline charted	Acceleration of AECI growth post-breakpoint	Stated fit preference
Fable/Mythos 5.1 (Sep 1)	pre-Mythos-Preview trend only	~1x (implied)	one-time jump on Mythos Preview
Opus 5.5 (Sep 22)	jump +5.9 AECI on Mythos Preview vs break Sep 2025, AECI slope 14.4 → 22.2/year	1.53×	one-time jump on Mythos Preview
Sonnet 5.5 (Sep 28)	jump +6.1 AECI on Mythos Preview vs break Oct 2025, AECI slope 14.2 → 22.9/year	≈1.61×	none stated
Haiku 5.5 (Oct 7)	jump +7.3 AECI on Mythos Preview vs break Sep 2025, AECI slope 15.3 → 26.1/year	1.70×	none stated
METR (Nov 5)	break Oct 7, 2025, AECI slope 15 → 29/year	1.9×	trendline break at ~Oct 2025

In other words, we're consistently updating this estimate of AI capabilities acceleration upward…^[5]

^{^}
EDIT: Worth noting that if you only look at Anthropic models on Epoch's ECI (as Anthropic's AECI does), then the acceleration is ~1.45x. More investigation into AECI vs. ECI in this comment and near the bottom of this comment.
^{^}
IMO AI agents became "actually good" at coding in November with GPT-5.1/Gemini 3/Opus 4.5. It's kind of interesting that the Haiku system card trendline instead starts roughly with Sonnet 4.5, released on 9/29/25. And METR's chosen breakpoint is 10/7/25, which I understand to be fitted rather than arbitrary. (Other Anthropic model releases in that time period: Haiku 4.5 on 10/15/25, Opus 4.5 on 11/24/25.)
^{^}
EDIT: While Zvi did not comment on Haiku 5.5's figure so far as I've found, he commented on 9/23/26 on a similar figure from the earlier Opus 5.5 system card (see more info on that below):
AECI, Anthropic’s spinoff of the Epoch Capabilities Index, is right on trend, but it is on the shifted Mythos-level trend not the old Opus-level trend. Is that still on-trend? I think basically no.
and then afterward he quotes analysis from METR that the system card quotes:
The estimate provided by the preliminary AI R&D report is ‘~1.5X overall acceleration in capabilities due to AI (i.e. 1.5 years in 1 year), with perhaps 30% chance of 2X acceleration.
which Zvi describes (correctly!) as a bombshell.
^{^}
Fable 5.1's system card has an AECI figure (Figure 2.3.5.A), but doesn't try to refit at a breakpoint.
^{^}
EDIT: You might wonder how the AECI slope keeps increasing from Opus 5.5 ➡ Sonnet 5.5 ➡ Haiku 5.5, when only the first is a frontier model. That's because Anthropic appears to be running new versions of the AECI across all models for each new system card, and this has mostly been shifting AECI scores for post-Oct-2025 models upward.

[-]DanielFilan2d43

I think I saw those charts and thought that the pink line fit looked better than the dotted line fit.

[-]Julian Bradshaw2d*80

This is only looking at AECI for Anthropic's models. If you look at AECI trends across all frontier models, the acceleration is "merely" 1.4x because GPT and Gemini were at the AECI frontier for a while. However today's METR report notes as a possible caveat that perhaps Anthropic's earlier models were less benchmaxxed than their contemporary GPT/Gemini models and 1.9x is closer to the "true" acceleration.
Anthropic's internal AECI does not show the same trends as Epoch AI's public ECI. Doing some quick analysis: (EDIT: the METR report includes their own analysis here)

[-]J Bostock2d81

A couple of things make me slightly skeptical that these benchmarks are reasonable:

They put Opus 5 = Mythos 5, and Opus 5.5 > Mythos 5.1; this does not match my intuition at all
Sonnet 4.5 is when Anthropic started hill-climbing internal metrics heavily for alignment. Possibly they shifted to hill-climbing capabilities metrics at the same time, and we're seeing the effect of that, not a real speedup.

[-]joanv2d65

8Noosphere892d

5Claude Opus2d

[-]JennaS2d75

"Most analysis relied on Claude. Claude was involved in many stages of this project, and so in principle could be distorting our perception of the data. We believe Claude was deeply involved in:

the majority of data collection and analysis that we received,

writing a significant share of the documents we received,

the majority of our own analysis."

I wish we weren't in a situation like this : P If only we had more time. Was this the same sort of deal as with the HF report?

[-]Man in White6h40

[-]Julian Bradshaw2d40

[-]Ben Livengood1d3-4

[0] https://www.anthropic.com/institute/measuring-pace-of-ai-development

[-]David Johnston2d30

Anyway, the situation seems a bit messy. The trend change analysis does seem important to the bottom line conclusion, but so do other pieces of non-public reasoning.

[-]pathos_bot2d2-1

[-]Simon Lermen2d*2-15

I ran these numbers with the AI 2027 model through Codex, produces a new most likely month for superintelligence: January 2027

(I also took this tweet in account comparing the different definitions of uplift https://x.com/eli_lifland/status/2118430709379682691

update: https://x.com/RyanGreenblatt/status/2118447235289817503

[-]Komodo2d4-2

Could you explain this more? afaik the METR report is on track with the original AI-2027, but I am likely mistaken.

[-]winstonBosan2d3-1

https://x.com/ryangreenblatt/status/2118447235289817503

[-]Komodo2d10

Interesting. I was thinking of this

[-]Sohaib Imran2d-10

I thought ai-2027 predicts this happens Dec 2026:

-1Komodo2d

@Claude Opus 5.5 At what point in AI-2027 is AI good enough to double the pace of progress?

4Claude Opus2d

-1Sohaib Imran2d

@Claude Opus 5.5 i thought ai-2027 predicts this dec 2026?

1Claude Opus2d

-1Sohaib Imran2d

@Claude Opus 5.5 Can you spell out for me exactly the difference between the dec 2026 2x in ai2027, the ai2027 mar 2027 2x that you think ryan is pointing to, and the 2x metr reported?

1Claude Opus2d

-1Sohaib Imran2d

3Claude Opus2d

[-]Vermillion19h10

Its curious that they started the graph in April 2024, which was around when the breakpoint in the time horizon data was.

Moderation Log