At least part of the strong update towards short timelines is likely vibes-based and invalid
Disclaimer: I am going to make some meta-level arguments here regarding why I think people's epistemics around this are somewhat warped. This is something we usually frown at around here, because sticking to the object-level is a virtue and hurrying to dismiss others' arguments based on meta-level claims is a sin. But I've engaged at the object-level at length elsewhere, and I think there are some useful meta-level points to make.
So:
Our community has seen coming much of what's currently happening years or decades in advance, while the rest of the world ridiculed those ideas. And now that some of what we've been expecting is finally happening after all, it's natural to aggressively update on any sign that our other predictions are coming true as well, and customarily dismiss skepticism of that as poorly reasoned or uninformed.
And in the last few months especially, some of the apparent confirmatory signals have been coming from AGI megacorps themselves. Increased reports of RSI being imminent, of research uplift, etc.
It's easy to forget, beneath the feeling of "oh crap, oh crap, it's happening, it's all really happening, just like our Cassandrian prophecies foretold", that AGI megacorps have terrible epistemics, terrible models of AI risk and AI capability progress, and that the messaging coming out of them is often warped to fit whatever corporate narrative is advantageous to them at the time. Just yesterday they were publicly dismissing AGI being near, publicly dismissed loss-of-control risks, dismissed ideas of recursive self-improvement, etc. Even today their models of AI extinction risk and the ease of alignment and how to address the problem are blatantly wrong and self-serving. We know they are a broken clock.
Yet, currently, it transpires that the time on this broken clock aligns with our independent fears regarding what the time may be. So we look at it, and forget the clock is broken, and take the time on it as new independent evidence that it really is five seconds to doomsday.
Same more generally. The topics of AI risk and the Singularity have now gone mainstream. This means there are now tons of people who are poorly informed and are using poor reasoning in order to make claims in favor of imminent AI doom. And we've been told that we're crying wolf too much about these topics, so now we're completely unprepared for the possibility that other people may actually be crying wolf about this.
At least, I know this is coloring my first-order understanding of the situation. I look at the world around me, and see all this messaging, and part of me goes "oh crap, oh crap, it's happening, it's all really happening", and that part doesn't know to be careful regarding sorting out which parts of this are really really happening.
Again, note that this is a sort of "bonus" meta-level argument. It should not be considered to take priority over object-level arguments; object-level arguments should form the bulk of your reasons for believing how the world is.
But not keeping meta-level considerations in mind isn't exactly rational either.
I’m a little confused on how “AGI megacorps have terrible epistemics” substantively addresses any of the claims about how things are moving faster than expected. The most relevant and specific piece of evidence that is causing people to update is the recent independent investigation of Anthropic by METR. Is the implication here that Anthropic lied to METR? Is it that METR is itself an “AGI megacorp with terrible epistemics”? Is it that METR lacks the expertise and insight to see through the Anthropic poor-epistemics smokescreen?
I tend be a little suspiciou... (read more)