This is a special post for quick takes by Vladimir_Nesov. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Comment Permalink
There likely won't be an 8-rack scale-up Nvidia system ready for the 2027 buildout after all (contrary to what I speculated), and even for the 2028 buildout it won't be offered in important quantities, suggests the SemiAnalysis report [1] posted after GTC 2026. If this is the case, then large scale-up worlds in the Nvidia buildout will follow the GTC 2025 timeline, with the first major change compared to GB200/GB300 Oberon racks (14/20 TB of HBM4) being the Rubin Ultra Kyber racks (147 TB of HBM4E), with the full-scale buildout in 2028.
Google TPUs will keep their advantage in hypothetical models with tens of trillions of parameters until 2028, and we might soon observe from Anthropic's rumored larger-than-Opus Claude 5 model whether that's likely to become an important class of models before 2028.
Specifically, they say it's 8x Oberon for racks with 72 Rubin Ultra packages, 4 compute dies per package, shipped starting in 2027 (rather than 72 non-Ultra Rubin packages, 2 compute dies per package, shipped starting in 2026). And that the multi-rack scale-up will be uncomfortably expensive, so unlikely to be shipped in volume. Since Kyber racks of Rubin Ultra should be out at about the same time, it doesn't seem crucial that this system is offered at all, other than as an early peek at the Feynman 8x Kyber (NVL 1152) systems of 2028-2029. ↩︎
2