Where is the hard evidence that LLMs are useful?

Has anyone seen convincing evidence of AI driving developer productivity or economic growth?

It seems I am only reading negative results about studies on applications.

https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/

https://www.lesswrong.com/posts/25JGNnT9Kg4aN5N5s/metr-research-update-algorithmic-vs-holistic-evaluation

And in terms of startup growth: 

https://www.lesswrong.com/posts/hxYiwSqmvxzCXuqty/generative-ai-is-not-causing-ycombinator-companies-to-grow

apparently wider economic measurements are not clear?

Also agency still seems very bad, about what I would have expected from decent scaffolding on top of GPT-3:

https://www.lesswrong.com/posts/89qhQH8eHsrZxveHp/claude-plays-whatever-it-wants
 

(Plus ongoing poor results on Pokémon, modern LLMs still can only win with elaborate task-specific scaffolding)

Though performance on the IMO seems impressive, the very few examples of mathematical discoveries by LLMs don’t seem (to me) to be increasing much in either frequency or quality, and so far are mostly of type “get a better lower bound by combinatorially trying stuff” which seems to advantage computers with or without AI. Also, again, even that type of example is rare, probably the vast majority of such attempts have failed and we only hear about a few successful ones, none of which seem to have been significant for any reason other than coming from an LLM.

I increasingly suspect a lot of the recent progress in LLMs has been illusory, from overfitting to benchmarks which may even leak to the training set (am I right about this?) and seeming useful, and METR is sufficiently good at their job that this will become apparent in task length measurements before the 8 hour mark.

I’m trying to make belief in rapid LLM progress pay rent, and at some point benchmarks are not the right currency. Maybe that point is “not yet” and we see useful applications only right before superintelligence etc. but I am skeptical of that narrative; at least, it does little to justify short timelines, because it leaves the point of usefulness to guesswork.