Eigenbraid

The LLM Revolution, Part 2 My previous post covered what LLMs are getting good at, but I also think it's important to survey their current limitations (as of July 2026). I don't think ASI is a big threat until most, if not all, of these issues are resolved. The rate of progress here is thus again a good barometer for whether we're in for faster or longer timelines. Currently, I struggle to imagine today's models succeeding at world domination. They still can't manage 99% uptime, and occasionally hallucinate a country. But it's a lot easier to imagine that we find a way to overcome those limitations - at which point word domination starts to worry me more and more. Reliability This is the biggest obstacle, but also the one with the clearest improvement. METR currently focuses primarily on 50% tasks success rates, which are impressive but require a lot of human verification. The bottleneck becomes quality, and at 50% reliability you can't accelerate that process much. Conversely, the METR charts for higher reliability generally lag the 50% success rate by a fairly stable window. Toggling the graphs between 50% and 80% success rate gets you two slightly differently scaled straight lines. The current cutting edge is Claude Mythos, with a 99% success rate on 4-minute tasks, 80% around 3 hours, and 50% around 17 hours. ChatGPT 4, released three years ago, had a 29% score for 4-minute tasks. That means today's "coin flip" is tomorrow's 99%. Gains here also compound in an interesting way: if your system has a 10% chance of making a mistake, then a second pass of equal accuracy mathematically produces a 90% chance of finding that mistake - that brings you down to ~1% error rate, albeit at double the cost (in both tokens and time taken). This doesn't always hold up, of course - sometimes verification is easier, and sometimes it's much harder. This is one part of why we see such rapid progress on easily verified domains. Long-Term context LLMs can fluidly switch betwe