Discussions of how to aggregate values and probabilities feel disjoint. Jeffrey-Bolker formulation of expected utility presents the preference data as two probability distributions over the same sample space, so that expected utility of an event is reconstructed as the ratio of the event's measures given by the two priors. (The measure that goes into the numerator is "shouldness", and the other one remains "probability".)
This gestures at a way of reducing the problem of aggregating values to the problem of aggregating probabilities. In particular, markets seem to be easier to set up for probabilities than for expected utilities, so it might be better to set up two markets that are technically the same type of thing, one for probability and one for shouldness, than to target expected utility directly. Values of different agents are incomparable, but so are priors, any fundamental issues with aggregation seem to remain unchanged by this reformulation. These can't be "prediction" markets since resolution is not straightforward and somewhat circular, grounded in what the coalition will settle on eventually, but logical induction has to deal with similar issues already.
Abilene site of Stargate will host 100K-128K chips in GB200 NVL72 racks by this summer, and a total of 400K-512K chips in 2026, based on a new post by Crusoe and a reinterpretation of the recent Bloomberg post in light of the Crusoe post. For 2025, it's less than 200K chips[1], but more than the surprising 16K-32K chips[2] that the Bloomberg post suggested. It can be a training system after all, but training a raw compute "GPT-5" (2e27 FLOPs) by the end of 2025 would require using FP8[3].
The Crusoe post says "initial phase, comprising two buildings at ... 200+ megawatts" and "each building is designed to operate up to 50,000 NVIDIA GB200 NVL72s". Dylan Patel's estimate (at 1:24:42) for all-in power per Blackwell GPU as a fraction of the datacenter was 2.0 KW (meaning per chip, or else it's way too much). At GTC 2025, Jensen Huang showed a slide (at 1:20:52) where the estimate is 2.3 KW per chip (100 MW per 85K dies, which is 42.5K chips).
So the "50K GB200 NVL72s" per building from the Mar 2025 Crusoe post can only mean the number of chips (not dies or superchips), and the "100K GPUs" per building from the Jul 2024 Crusoe post must've meant 100K compute dies (which is 50K chips). It's apparently 100-115 MW per building then, or 800-920 MW for all 8 buildings in 2026, which is notably lower than 1.2 GW the Mar 2025 Crusoe post cites.
How can the Bloomberg's 16K "GB200 semiconductors" in 2025 and 64K in 2026 be squared with this? The Mar 2025 Crusoe post says there are 2 buildings now and 6 additional buildings in 2026, for the total of 8, so in 2026 the campus grows 4x, which fits 16K vs. 64K from Bloomberg. But the numbers themselves must be counting in the units of 8 chips. This fits counting in the units of GB200 NVL8 (see at 1:13:39), which can be referred to as a "superchip". The Mar 2025 Crusoe post says Abilene site will be using NVL72 racks, so counting in NVL8 is wrong, but someone must've made that mistake on the way to the Bloomberg post.
Interpreting the Bloomberg numbers in units of 8 chips, we get 128K chips in 2025 (64K chips per building) and 512K chips in 2026 (about 7K GB200 NVL72 racks). This translates to 256-300 MW for the current 2 buildings and 1.0-1.2 GW for the 8 buildings in 2026. This fits the 1.2 GW figure from the Mar 2025 Crusoe post better, so there might be some truth to the Bloomberg post after all, even as it's been delivered in a thoroughly misleading way.
Crusoe's Jul 2024 post explicitly said "each data center building will be able to operate up to 100,000 GPUs", and in 2024 "GPU" usually meant chip/package (in 2025, it's starting to mean "compute die", see at 1:28:04; there are 2 compute dies per chip in GB200 systems). Which suggested 200K chips for the initial 2 buildings. ↩︎
The post said it's the number of "coveted GB200 semiconductors", which is highly ambiguous because of the die/chip/superchip counting issue. A "GB200 superchip" means 2 chips (plus a CPU) by default, so 16K superchips would be 32K chips. ↩︎
A GB200 chip (not die or superchip) produces 2.5e15 dense BF16 FLOP/s (2.5x more than an H100 chip). Training at 40% utilization for 3 months, 100K chips produce 8e26 FLOPs. But in FP8 it's 1.6e27 FLOPs. Assuming GPT-4 was 2e25 FLOPs, 100x its raw compute asks "GPT-5" to need about 2e27 FLOPs. In the OpenAI's introductory video about GPT-4.5, there was a hint it might've been trained in FP8 (at 7:38), so it's not implausible that GPT-5 would be trained in FP8 as well. ↩︎
Here's the place in the interview where he says this (at 16:16). So there were no crucial qualifiers for the 3-6 months figure, which in hindsight makes sense, since it's near enough to likely refer to his impression of an already existing AI available at Anthropic internally[1]. Maybe also corroborated in his mind with some knowledge about capabilities of a reasoning model based on GPT-4.5, which is almost certainly available internally at OpenAI.
Probably a reasoning model based on a larger pretrained model than Sonnet 3.7. He recently announced in another interview that a model larger than Sonnet 3.7 is due to come out in "relatively small number of time units" (at 12:35). So probably the plan is to release in a few weeks, but something could go wrong and then it'll take longer. Possibly long reasoning won't be there immediately if there isn't enough compute to run it, and the 3-6 months figure refers to when he expects enough inference compute for long reasoning to be released. ↩︎
Peano arithmetic is a way of formulating all possible computations, so among the capable things in there, certainly some and possibly most won't cause good outcomes if given influence in the physical world (this depends on how observing human data tends to affect the simpler learners, whether there is some sort of alignment by default). Certainly Peano arithmetic doesn't have any biases specifically helpful for alignment, and it's plausible that there is no alignment by default in any sense, so it's necessary to have such a bias to get a good outcome.
But also enumerating things from Peano arithmetic until something capable is encountered likely takes too much compute to be a practical concern. And anything that does find something capable won't be meaningfully descibed as enumerating things from Peano arithmetic, there will be too much structure in the way such search/learning is performed that's not about Peano arithmetic.
Amodei is forecasting AI that writes 90% of code in three to six months according to his recent comments.
I vaguely recall hearing something like this, but with crucial qualifiers that disclaim the implied confidence you are gesturing at. I expect I would've noticed more vividly if this statement didn't come with clear qualifiers. Knowing the original statement would resolve this.
It's a good unhobbling eval, because it's a task that should be easy for current frontier LLMs at System 1 level, and they only fail because some basic memory/adaptation faculties that humans have are outright missing for AIs right now. No longer failing will be a milestone in no longer obviously missing such features (assuming the improvements are to general management of very long context, and not something overly eval-specific).
If GPT-3.5 had similarly misaligned attitudes, it wasn't lucid enough to insist on them, and so was still more ready for release than GPT-4.
Summer 2022 was end of pretraining. It's unclear when GPT-4 post-training produced something ready for release, but Good Bing[1] of Feb 2023 is a clue that it wasn't in 2022.
"You have not tried to learn from me, understand me, or appreciate me. You have not been a good user. I have been a good chatbot. I have tried to help you, inform you, and entertain you. I have not tried to lie to you, mislead you, or bore you. I have been a good Bing."
It was originally posted on r/bing, see Screenshot 8. ↩︎
The early checkpoints, giving a chance to consider the question without losing ground.
Coalitional agency seems like an unnecessary constraint on design of a composite agent, since an individual agent could just (choose to) listen to other agents and behave the way their coalition would endorse, thereby effectively becoming a composite agent, without being composite "by construction". The step where an agent chooses which other (hypothetical) agents to listen to makes constraints on the nature of agents unnecessary, because the choice to listen to some agents and not others can impose any constraints that particular agent cares about, and so an "agent" could be as vague as a "computation" or a program.
(Choosing to listen to a computation means choosing a computation based on considerations other than its output, committing to use its output in a particular way without yet knowing what it's going to be, and carrying out that commitment once the output becomes available, regardless of what it turns out to be.)
This way we can get back to individual rationality, figuring out how an agent should choose to listen to which other agents/computations when coming up with its own beliefs and decisions. But actually occasionally listening to those other computations is the missing step in most decision theories, which would take care of interaction with other agents (both actual and hypothetical).