A few months ago I gave a ~10% probability of TEDAI within 12 months; my intuition is now closer to ~30% if we go full speed ahead. Some of that was from the passage of time, but it's also about a ~2x update.[1]
For me the math and CS research progress is a bigger update than the METR/Anthropic reporting. I think we saw maybe a decade of progress on the hardest open problems in math and theoretical computer science with no sign of relevant capabilities slowing down. The ability to outperform large communities of human researchers at the task of designing and improving algorithms is a core part of an intelligence explosion, so I think that's a really important indicator.[2]
I also think it's useful to track accelerated progress on well-defined metrics (like pretraining or RL performance). Regardless of whether that progress transfers, it indicates that underlying technical developments are accelerating. And beyond that, I think "we haven't seen the pace of progress double yet" is a good argument that the singularity is likely at least 6-12 months away (at least for a trend-extrapolator like me). So once it's happened it's a lot harder to be confident that you won't see something crazy happen in the near term, which naturally lead to a vibes shift even if it still leaves significant probability on things take a while.
So maybe my main disagreement with you is that it looks fairly likely that a significant period of self-sustaining acceleration in AI R&D (which is suggested both by the massive math progress and early signs of rapid adoption+acceleration from AI labs) would likely lead to ASI. This is a messy discussion I can't do justice to, but here are some things I believe:
(Eliezer has written a lot about these dynamics, which often comes across as him saying "intelligence is magic." I always thought it was a bit of a distraction, but now on the eve of the singularity seeing that this is the crux of the argument with so many people I have much more sympathy for his choice of emphasis.)
That's maybe all consistent with what you said. I agree there is no trivial way to extrapolate to ASI from something like a 2x acceleration in AECI progress (though the fact that AECI is built on verifiable tasks is far from the biggest problem with that approach!). And I also agree that there's still a meaningful probability of no ASI in the next few years. It just seems like a fairly likely possibility at this point, and I think the vibes more broadly reflect that.
And although ~30% feels like the right all-things-considered estimate to me, it's obviously relevant that I've underestimated the pace of progress multiple times before and never seriously overestimated it. I think I was broadly right that "crazy smart AI is preceded by slightly less smart AI, and the gap is a few years rather than a few months," but everything has happened ~2x faster than I thought even after updating on how the last few years had already been ~2x faster than I thought. So outside view it wouldn't be crazy to just assume the next update will be in the same direction.
It's also particularly impactful because we know that AI systems are already improving rapidly at tasks like running and debugging experiments, and I expect them to soon be very superhuman on the task of predicting the result of experiments in advance, and so it's really not clear what you would point to as the candidate blocker.
And even if we ignored the possibility of accelerating progress on deep learning, we'll soon be in the regime where if there was a much cleverer way to build AI systems that humans haven't thought of (which honestly seems more likely to not) then an AI working for a week is more likely to find it than the last 60 years of human progress. At that point I think there's a significant probability of something even crazier happening.
Similar to other people's shortform feeds, short stuff that people on LW might be interested in, but which doesn't feel like it's worth a separate post. (Will probably be mostly cross-posted from my Facebook wall.)