This is a special post for quick takes by JustisMills. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
I haven't noticed anyone else come out and say it here, and I may express this more rigorously later, but, like, GPT-5 is a pretty big bear signal, right? Not just in terms of benchmarks suggesting a non-superexponential trend but also, to name a few angles/anecdata:
It did slightly worse than o3 at the first thing I tried it on (with thinking mode on)
It failed to one-shot a practical coding problem that was entirely in javascript, which is generally among its strengths (and the problem was certainly in principle solvable)
It's hallucinating pretty obviously when I give it a 100 page or so document to read and comment on (it references lots from the document, but gets many details wrong and overfixates on others)
It's described as an automatic family of models that naturally picks the best for the situation, which seems like exactly what you'd do if nothing else was really giving you the sauce
The main reddit reaction seems to be that the demo graphs were atrocious, which is not exactly glowing praise
All the above paired with the fact that this is what they chose to call GPT-5, and with the fact that Claude's latest release was a well-named and justified 0.1 increment
I'm largely with Zvi that even integrating this stuff as it already exists into the economy does some interesting stuff, and that we live in interesting times already. But other than what's already priced in by integrations and efficiency optimizations, progress feels s-curvier to me today than it did a week ago.
Broadly agreed. My sense is that our default prediction from here should be to extrapolate out the METR horizon length trend (until compute becomes more scarce at least) with somewhere between the more recent faster doubling time (122 days) and the slower doubling time we've seen longer term (213 days) rather than expecting substantially faster than this trend in the short term.
So, maybe I expect a ~160 day doubling trend over the next 3 years which implies a 50% reliability horizon length of ~1 week in 2 years and ~1.5 months in 3 years. By the end of thi... (read more)
same, I was surprised. I think my timelines ... ooh man I don't know what my mental timeline predictor did but I don't think "get longer" is a full description. winter is a scary time because summer might come any day
3Cole Wyeth
Mostly agree but also the slower (pre-reasoning model) exponential on task length is certainly not falsified and even looks slightly more plausible.