Quick Takes

ACX on the Frankfurt School is, perhaps, the best summary of the Frankfurt School of its length that exists anywhere. Few quibbles: 1. Weird to write about the Frankfurt school without mentioning Freud. The split in their influence is probably 60/40 Marx/Freud, so this seems like a big omission. Maybe Freud is too big a can of worms for Scott (or the secondary author he read) to introduce him here, and you can tell enough of the story without him. Ultimately I think this choice was reasonable. 2. I think Scott conflates the form and purpose of their work too much. That is, you can do witchy art criticism of the kind they ~invented without yourself being a Marxist, and this method/approach is perhaps more influential than their actual Marxism (since they spent more words on it, except for Marcuse). As Scott points out early on, many of those influenced by them would decry the affiliation. I claim this is usually because they’re taken to be either too Marxist or not Marxist enough by those who adapt their method without adopting their politics (which is most of those influenced by them). 3. He says upfront that they were weird and mystical for obscurantist reasons, and for intellectual-lineage German reasons, and later criticizes them as though their true form of belief were more-or-less as mystical as it might naively appear. I think the mistake is kind of obvious here (not saying there’s no mysticism, but your attitude toward how mystical they are shouldn’t subsume your interpretation of them, any more than Kant’s mysticism should subsume your interpretation of him - probably less). 4. Benjamin is far and away the most mystical of them, to such an extent that many readers cite his comparative mysticism as The Reason they (dis)prefer his work vs others in the school. I think that the mysticism critique is focused on quotes from him is evidence for the weakness of the point as generalized to the whole school. 5. Probably the Frankfurt school would just bite the

Some infographics by Steven Byrnes I've wanted to point people to, but take ~forever to find because he's so prolific, collected here for my own convenience. (I forgot to include the links to each, FML. I also mixed related-ish infographics from different sources in the same section a lot.) What AGI is and isn't, and why LLMs aren't it A frequent point of confusion is the word “General” in “Artificial General Intelligence”: * The word “General” DOES mean “not specific”, as in “In general, Boston is a nice place to live.” * The word “General” DOES NOT mean “universal”, as in “I have a general proof of the math theorem.” An AGI is not “general” in the latter sense. It is not a thing that can instantly find every pattern and solve every problem. Humans can’t do that either! In fact, no algorithm can, because that’s fundamentally impossible. Instead, an AGI is a thing that, when faced with a difficult problem, might be able to solve the problem easily, but if not, maybe it can build a tool to solve the problem, or it can find a clever way to avoid the problem altogether, etc. Consider: Humans wanted to go to the moon, and then they figured out how to do so, by inventing extraordinarily complicated science and engineering and infrastructure and machines. Humans don’t have a specific evolved capacity to go to the moon, akin to birds’ specific evolved capacity to build nests. But they got it done anyway, using their “general” ability to figure things out and get things done. So for our purposes here, think of AGI as an algorithm which can “figure things out” and “understand what’s going on” and “get things done”, including using language and science and technology, in a way that’s reminiscent of how most adult humans (and groups and societies of humans) can do those things, but toddlers and chimpanzees and today’s large language models (LLMs) can’t. This image is poking fun at  that “there is no such thing as Artificial General Intelligence”. (Image sources: ,)Th

By the end of 2030, models with up to 2-3 quadrillions of total params will be practical (but with 30x sparsity, only about 1.3 quadrillions might be actually used). [1] The capacity claim follows from Nvidia roadmap for 8x Kyber systems for Feynman (NVL1152) and the HBM roadmap, which for HBM5 has 16-Hi stacks of 5 GB per DRAM die in 2029 (while HBM4E is capped at the same 4 GB per DRAM die in 16-Hi stacks as HBM4 in Rubin). This is on the same timeline that has HBM4 in 2026, which first appears in Nvidia Rubin (which comes out in 2026, with most of the buildout going online over 2027). In Blackwell and Rubin, each compute die has 4 stacks of HBM, thus with 16-Hi stacks with 4 GB per DRAM die (HBM4E) we get 256 GB per die, and with 5 GB per DRAM die (HBM5) we get 320 GB per die. A Kyber rack hosts 576 compute dies, thus 147 TB per Rubin Ultra Kyber rack (the buildout for this system is mostly over 2028). Feynman first comes out in 2028, so it won't use HBM5 in the first year, but if the 8x Kyber (NVL1152) systems are released in the first year, they will have the capacity of 1.18 PB per scale-up system (the buildout is over 2029). In 2029, the second year of Feynman (which might get called Feynman Ultra), HBM5 is likely available, and so 8x Kyber for Feynman Ultra will have 1.47 PB per scale-up system (the buildout is over 2030). Assuming 2-4 scale-up systems per deployment instance, with up to half of capacity used for weights, that's up to 1.2-2.4e15 FP8 total params for 8x Kyber Feynman, or 1.5-2.9e15 for Feynman Ultra. If a leading AI company has about 40 GW of first-party compute by the end of 2030 (which follows under some assumptions from the SemiAnalysis estimates of global AI compute), and uses about 10 GW of Feynman capacity for pretraining (in FP8, for 3 months, with 40% FLOP/s utilization), then assuming Feynman produces about 14e15 FP8 FLOP/s per compute die (which follows from extrapolating Rubin performance given the TSMC process change and possibl

I got quite sad yesterday staring into the abyss, confronting the reality of current scaling trends, and feeling like we live at thhe end of time Edit: I am doing fine mentally, emotionally etc. No suicidal tendencies. Thanks everyone who was concerned :)

Steering reasoning models via grader fine-tuning Hot take: reasoning models trained with diverse RL across diverse graders will be steerable by fine-tuning them to assume a narrow distribution of graders. This is analogous to how pretrained models can be instruct-tuned to assume a persona. The analogy: pretraining teaches a model to simulate and select a writer from a distribution of writers. RL (reasoning training) similarly teaches it to simulate and select a grader from a distribution of graders. So you could steer an LLM by specifying both a persona and a grader within the training distribution. The upshot is that at the end of RL training, you'd want to train on a narrow distribution of graders that promote the goals you actually care about. This might mean using your strongest, most expensive graders and/or your least gameable training environments at that final stage. A plausible (simplified) pipeline: pretraining → HHH fine-tuning → RL(VR) → omniscient-grader fine-tuning.

Dear paper authors. PLEASE FOR THE LOVE OF GOD PUT MODEL SIZES IN THE PAPER TEXT AND FIGURE LEGENDS. Seriously. Specifying Gemma-3-12B isn't that many more characters than Gemma-3. I shouldn't have to trawl your methods for this. Gemma-3-27B is closer to Qwen-2-32B in basically everything I care about than it is to Gemma-3-1B. In fact the model size is the most important thing about the model! If you just said "27B" that would give me more information than "Gemma-3", yet somehow you have managed to provide 20% of the information in 80% of the characters! How did we end up with this particular convention?

I'm pretty sure the "to speak plainly with the people, use words of Anglo rather than Latin origin" point is itself an anachronism and affectation. I think most decent writers have a pretty good "vibe" for which words are common vs not, already. But if you're unsure, statistical frequency tests are a better proxy for familiarity than etymology. __ Attempted rewrite: I reckon the rede "to speak straight with the folk, reach for words with Anglo root, rather than that of the Walnut Folk" is itself a thing out of its own time, and a put-on of the highest kind. I think most writers worth their salt already have a pretty good ken for which words are everyday and which aren’t. But if you can't rightly tell, a reckoning of how oft a word shows up is a better gauge for how couth it is than a word’s wellspring.