I have been checking LessWrong every few hours over the last day or two to see if anyone is talking about how good Astra is at coding and computer use. It's a leap from Fable 5.1. The idea that AGI isn't already here is getting harder and harder to defend.
Like all LLMs it's still pretty terrible at high-level thinking. Surprisingly bad, for how good it is at individual tasks. I sometimes wonder if that has something to do with [EDIT: removing hypothesis for why LLMs have lower capability than they ought to], I'm not sure. Specific skills like vision/artistic taste are weak still too, though arguably approaching the average human level.
I'm having a hard time describing what makes it such a leap for me. I just feel like the number of tasks I can give it and have it do a good job without getting confused is greater. I trust it more to not screw up, and it can work for longer without oversight. This is true to the point where it seems like a leap over what existed before.
As an example, I made a little game on the weekend to test Astra's abilities. The game had performance issues due to a lot of 3D physics objects being on screen, and I told Astra to get the performance up from around 1 FPS to my ideal of 250. It worked for 2 hours, doing a lot of optimizations, including rewriting expensive functions in C. I had to make one suggestion (that it optimize the assets by reducing vertices) but otherwise it got to 300+ FPS on its own. It even handled baking high poly 3D models onto low poly optimized assets without my input (using Blender MCP) Any other AI model and I'd have been going back and forth with it about each optimization it was going to do, verifying it worked and didn't make things worse, etc. The whole time it was opening the game, testing performance itself, etc.
6Archimedes
I find Astra a definite step up in agentic thinking, especially for math research. It still has poor long-term research taste, but it is significantly less myopic than Sol for shorter to intermediate assignments. It can productively work in a loop for longer without needing steering.
Yes the poor long term taste is a thing still for me too, though I'm not doing research. I think LLMs just have... Actually maybe I'll refrain from talking about the reasons why I think LLMs are below where they could be on a public forum. But I agree they're still lacking in long-term judgment.