StanislavKrym's Shortform

StanislavKrym

This is a special post for quick takes by StanislavKrym. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

I looked up EpochAI's list of benchmarks. The very poor ECI of Grok 4.5 as opposed to GPT-5.5 seems to be a result of xAI not caring about math. Additionally, Grok 4.6 seems to be closer to Opus 4.8 with respect to ARC-AGI-3, if not outright outperforming Opus, as the actual ARC-AGI-3 score suggests. Does it mean that xAI is less behind than we think? I guess that Zvi will have to write something like "Grok 4.6 is three, not six, mounths behind. Stop xAI to hell!"

P.S. The same issue seems to apply to Meta's Muse Spark, except that it has even less evaluated benchmarks.

Also, SpaceX has caught up to Meta in credibly having enough compute in 2027-2028 ^[1] to stay in the game, if either of them can assemble a functional model development team. As Google illustrates, it's not easy to do that (even when you have some of the best people), but as OpenAI and Anthropic illustrate, it's not so difficult that it can't be replicated. Muse Sparks are probably small enough models that their non-frontier performance doesn't count as evidence that they're not well-made (and that a Mythos-sized Muse model won't have Mythos-lev... (read more)