My takeaways from METR's Independent Assessment of AI R&D Acceleration at Anthropic:

1: Anthropic and OpenAI employees were right about timelines (or at least closer to right than the general LessWrong consensus). Timelines seem short. (I haven't had time to figure out how short yet but it seems like things are going faster than AI 2027.)

Anthropic announced the same 2x acceleration number as METR in their S-1 and nobody took it that seriously because people (myself included) thought Anthropic tended to overhype things or be in a bubble. But they were just exactly right.

The information in the METR report makes timelines seem short because...

1.A: The best models are somewhat better than we thought, Mythos wasn't a one-off jump in capabilities.

You might have thought the acceleration was more like the green or even the purple line, but it's faster than both.

1.B. "We guess that a majority (perhaps ≈80%) of the current (October 2026) acceleration is due to AI. " I think many people thought the acceleration that got us to Mythos was from things like increased compute, not RSI feedback loops.

1.C: The benchmark scores are not overfit/gamed: METR looked at many sources of evidence to reach their conclusions, specifically:

1. Different methodological variants of internal metrics. We spent multiple weeks on these investigations, and this is our greatest source of uncertainty.
2. The beliefs of staff on the relevant teams about key internal evidence and estimates of acceleration.
3. Whether other parts of the model training pipeline could be biasing our estimates of acceleration, and where adjustments could be applied.
4. Whether Anthropic could be overfitting to internal metrics.

and not only was it all consistent with 2x acceleration, if anything "some of this evidence indicated that acceleration could be above 2x and increasing over time".

Reading the report made me reflect on how much clarity there is around the extra information folks inside AI companies have. I think I hadn't priced in that having all these clear benchmarks with new models might make it somewhat difficult to dramatically overestimate progress that's a few years out.

2: METR didn't identify any mitigating factor that might lengthen timelines; "We did not identify any convincing bottlenecks to continued rapid progress." I keep trying to make up stories for why timelines will be longer: maybe the fuzzy long-horizon tasks won't be automated, maybe there will be a hardware bottleneck. But if METR's right, then it seems like that's not the case and we should just expect the doublings to continue until we get superintelligence.

3: Anthropic isn't releasing all their frontier models. There are three new generations of model after Mythos that haven't been released.

4: Wow there's a lot of redactions. And if I didn't trust METR's judgement immensely, I would be a lot more hesitant to update so strongly on these findings. I also really wish Anthropic had released some of this information (especially the AECI scores) publicly proactively, instead of having to have METR coax it out of them.

Reading this report, I don't actually intuitively feel that shocked by any of the numbers. I think I should have updated faster sooner on timelines and was looking for excuses to believe they were longer, deferring too much, and afraid of looking dumb.