But I have met non-AI people who believe that the US government likely has access to secret models more powerful than the labs'.
People get a thrill from telling stories like that. Invariably they heard it from this guy who heard it from this guy who can't say where he heard it, but believe me...
The probability that any government entity in the world has independently developed anything even close to what the "labs" have is very small. It's not easy to hide that kind of compute, nor is it easy to get the necessary talent. The government can't spend a trillion dollars without anybody noticing, nor build giant data centers or huge state-of-the-art fabs.
Maybe they're ahead in very narrow areas directly related to their jobs: things like small models for specific military applications, or versions of the "labs"' LLMs fine-tuned for breaking security. But I doubt even that at this point.
There is a really good chance that they have access to "lab" models that normal people don't. Less restricted, and less enstupidified by "safety" training. But that's not the same as being ahead of the "labs".
They're probably not particularly far ahead in cryptography nowadays either. But at least cryptography is something the NSA actually had and has as a mission. They had by far the most resources on it, and first pick of the talent, and a big head start, and some hybrid of a gentlemen's agreement and a coercive arrangement that everything anybody outside invented was disclosed to them and nobody else, for a long time... in the last century. It was specialized military technology then. Now it's dual use, or indeed just plain general-purpose stuff. And no, they cannot break any of the ciphers in wide use now, assuming they're used correctly (which they've sometimes tried to undermine, but at best managed to delay).
In 1960 or 1970, the US Government, or at least the military arms, had a pretty large lead over everybody in all forms of "high tech". They got everything first and best. That hasn't been true for a long time.
There's obviously no truth to that claim. Labs absolutely have better models.
People in the AI space, myself included, seem to typically believe that the smartest models that exist today were trained by scaling labs.
But I have met non-AI people who believe that the US government likely has access to secret models more powerful than the labs'.
What is the best evidence either way? If (as I expect) the current evidence points towards today's smartest models having been trained by labs, what evidence should I look out for in the future that would indicate that the situation had changed?