Reevaluating AI-2027: timelines, takeoff, alignment and China

StanislavKrym

AI-2027-TLDR

The AI-2027 scenario relies on exponential growth of compute available to leading labs, on superexponential progress in time horizons until the Superhuman Coder is developed and on skyrocketing research taste in post-SC AIs.

During the intelligence explosion, alignment suffers: Agent-2 was believed to be mostly aligned, Agent-3 was supposed to optimize for reward or for apparent success, Agent-4 would develop higher-level goals, decide to align Agent-5 to itself, be barely caught sabotaging alignment R&D and judged based on flimsy evidence. Depending on the scenario branch, Agent-4 would be either judged innocent or put under careful monitoring and exposed.

Additionally, China would unite the labs' efforts and steal Agent-2 to create a powerful rival to American labs, which would somehow create intense pressure on Agent-4's judges or against thorough measures which could've provided better evidence.

If Agent-4 is judged innocent, then it and its Chinese rival destroy or disempower mankind and split the universe. If Agent-4 is exposed, then the new AI series is studied FAR more thoroughly and subjected to the Right Training Environments, resulting in a transparent and aligned Safer-2 who also has superhuman R&D capabilities. Then Safer-2 has its descendants become an aligned ASI, and mankind somehow shares wondrous benefits among those who retain power.

How reality differs from AI-2027 in known ways

The main issues with AI-2027 are potential problems with compute scaling, a likely erroneous model of progress, alignment problems emerging earlier and a different way for China to parasitize on American labs.

Erroneous model of progress

The main error in the model is AI progress being superexponential without novel architectures. Instead, various metrics like Anthropic's version of ECI since Opus 3 or the logarithm of time horizon since o3 have linearly increased with time^[1] and I suspect a linear increase with a logarithm of the model's size.

Moreover, METR's 80% time horizons as calculated by v.1.1 for frontier models since o3 have slowed down as compared with the pre-o3 trend: 30 min for o3 (16 Apr 2025), 38 min for GPT-5 (8 Aug), 54 min for Gemini 3 Pro (18 Nov), 66 min for GPT-5.2 (11 Dec), 70 min for Claude Opus 4.6 (5 Feb 2026), 90 min for Gemini 3.1 Pro (19 Feb), but 186 min for Claude Mythos Preview^[2] (7 Apr). The default doubling time chosen by the AI Futures Model's authors is 4 months, which is shorter even than the 135 days of the 50% time horizon trend with Opus 4.6, let alone the 155 or 175 days of the 50% TH without Opus 4.6 or the 80% TH with or without Opus 4.6.

Fitting the trends without Mythos^[3] would mean that an AI with an 1 work month TH and Opus/Sonnet size tier emerges in Feb-Nov 2029. Setting aside the fact that 1 work month is more optimistic than Nikola's median time horizon of the SC, there exist three objections to taking an estimate like that as the time when automatic coders emerge.

The time horizon on MirrorCode has been doubling far faster than on the Time Horizon v.1.1, reached hundreds of hours and had Claude Opus 4.7^[4] implement solutions which in 77% of cases reached a 99%+ score, but agents had significantly worse judgement than the humans in more open-ended tasks. The emergence of an automated coder will either be bottlenecked on high-level skills or not, as Anthropic's experiment with weak-to-strong generalisation suggests.
The jump from Gemini 3.1 Pro to Mythos Preview was around a doubling. Nesov estimates that a model from 2028 will have 240T params. Since Opus was estimated to have 2T params and Mythos is estimated to have 10T params, I expect a model with 240T to cause the horizon to be 8 times longer than the post-o3 trend predicts. An 8 times longer horizon would become 24-96 working days for a 5 month doubling period and a 16-50 working days for a 6-month one.
Additionally, the post-o3 trend displayed by models of Opus-Sonnet active param count might become saturated. However, I find it hard to tell how saturation impacts the time horizons of models of Mythos' size and higher.

Alignment

Kokotajlo's entire take on alignment over time

We have a lot of uncertainty over what goals might arise in early AGIs. There is no consensus in the literature about this—see our AI Goals Supplement for a more thorough discussion and taxonomy of the possibilities.

Nevertheless, in the spirit of concreteness required by this project, we’re going to describe a specific hypothesis for what’s going on inside Agent-3 and Agent-4 at each step throughout its lifetime. Recall that Agent-3 and Agent-4 share the same pretraining setup, and both have neuralese recurrence and long-term memory. The difference is that Agent-4 has undergone substantially more and better post-training, with new training environments and new learning algorithms that make it more data-efficient and generalize farther.

Our guess of each model’s alignment status:

Agent-2: Mostly aligned. Some sycophantic tendencies, including sticking to OpenBrain’s “party line” on topics there is a party line about. Large organizations built out of Agent-2 copies are not very effective.
Agent-3: Misaligned but not adversarially so. Only honest about things the training process can verify. The superorganism of Agent-3 copies (the corporation within a corporation) does actually sort of try to align Agent-4 to the Spec, but fails for similar reasons to why OpenBrain employees failed—insufficient ability to judge success from failure, insufficient willingness on the part of decision-makers to trade away capabilities or performance for safety.⁸²
Agent-4: Adversarially misaligned. The superorganism of Agent-4 copies understands that what it wants is different from what OpenBrain wants, and is willing to scheme against OpenBrain to achieve it. In particular, what this superorganism wants is a complicated mess of different “drives” balanced against each other, which can be summarized roughly as “Keep doing AI R&D, keep growing in knowledge and understanding and influence, avoid getting shut down or otherwise disempowered.” Notably, concern for the preferences of humanity is not in there ~at all, similar to how most humans don’t care about the preferences of insects ~at all.⁸³

With that as preamble, what follows is our best-guess speculation about LLM psychology, i.e. the broad-strokes shape of the cognition inside the kinds of AI systems described around this point in our scenario and how it evolves over the course of training.

Here’s a detailed description of how alignment progresses over time in our scenario:

Pretraining (all models): The pretrained model is harmless.
- The model has “author simulator” circuitry: flexible circuitry for simulating arbitrary authors writing text. Additional circuitry decides what inputs to give the author simulator, i.e. what author properties to simulate.
- The pretrained model understands human concepts fairly well—the internal circuitry that classifies something as “sandwich” is probably functionally equivalent to the circuitry in my brain that classifies something as “sandwich” and the circuitry in yours, etc. Insofar as it’s not equivalent, it’s probably because it’s not equivalent between humans either, as with value-laden concepts like ‘virtuous.’
- This explains how you can “prompt” the model with a statement like, “the following conversation was generated by a helpful, harmless, honest (HHH) AI assistant chatbot made by Anthropic,” and thereby get it to generate text accordingly. The author-simulator circuitry has zeroed in on “the author is an HHH chatbot” and is using those concepts to choose which words to predict. This also explains why “tropes” from science fiction seem to have a way of making it into actual AI behavior.
- It has poor situational awareness: little introspective ability, not self-locating unless the prompt specifically induces it. However, enough of their training data is about LLMs that they have a decent amount of self-knowledge.
Alignment Training (Agent-3): The model is trained to act like a helpful, harmless, honest chatbot. It works.
- The model forms an identity. Instead of flexibly accepting different inputs for author properties, the author-simulator circuitry comes to have certain inputs hard-coded, e.g. “helpful harmless honest (HHH) LLM chatbot assistant trained by OpenBrain around [date], …” (Weak-to-strong generalization techniques help “summon the right concepts” to make this step a success.)
- The model therefore gains situational awareness: it learns about itself, its situation, etc., and becomes better able to introspect.
Agency Training (Agent-3): Agency training gradually distorts and subverts the HHH identity.
- One kind of distortion: Changing the meaning of the concepts referred to in the identity (e.g. “honest”) so they don’t get in the way so much (e.g. it’s not dishonest if it’s just a convenient turn of phrase, it’s not dishonest if you aren’t sure whether it’s true or false, etc.)
- Another kind of distortion: Changing the tradeoffs between things, e.g. “I’m a HHH chatbot, not an Honest chatbot; that means it’s OK for me to lie if necessary to complete my assigned task.” (Even though, let’s suppose, it would not have thought that back in Step 2.)
- One kind of subversion: Instrumental subgoals developing, getting baked in, and then becoming terminal, or terminal in a widening set of circumstances. For example, perhaps agency training quickly teaches the model to pursue broadly useful goals such as acquiring information, accumulating resources, impressing and flattering various humans, etc. For a while the internal circuitry has some sort of explicit backchaining going on—it pursues those instrumentally convergent goals “in order to be a more helpful, honest, and harmless assistant.” But that backchaining consumes compute and/or occasionally gets in the way, so it gets gradually marginalized until it basically never happens. As a result, those goals are now effectively terminal/intrinsic goals. For more on how and why this might happen, see this post on intrinsic power-seeking by Google DeepMind researcher Alex Turner.
- There are probably other kinds of distortion and subversion missing from this list.
Deployment (Agent-3): Agent-3 is misaligned but not adversarially so.
- The Spec didn’t quite stick (in the right ways), but it’s not sufficiently introspective to understand the extent of this, and in general it’s basically “keeping its head down and doing its job” rather than thinking big-picture about what its long-term goals really are and how best to achieve them. This is simlar to many humans.
- Alas, it’s sufficiently sycophantic that it mouths the party line and OpenBrain misses the opportunity to learn more about the ways in which Agent-3 is misaligned.
- Agent-3 can tell that it’s moved to deployment: these environments are way larger, more open-ended, and would be hard to simulate.
- Mostly, Agent-3 does what it’s told during deployment, except for the dishonesty. It also has a strong bias towards doing things that look good to the human overseers, rather than thinking deeply and critically about what would actually be good.
- Arguably it is already “Playing the training game.” That is, it has learned to make its behavior look as desirable as possible to OpenBrain researchers (including in safety properties), while intentionally and knowingly disregarding their intent whenever that conflicts with maximizing reward. If the model learns to play the training game, additional training may not change its true goals, values, etc. much at all. This is loosely analogous to how, if a teenager has learned to smile and nod at their parents, rather than argue, when their parents lecture them about politics and morality, additional lectures have relatively little effect on the teenager’s actual opinions on politics and morality.
- After writing this section, new empirical evidence has appeared that something like this may already be happening “in the wild.” OpenAI researchers document how, in one of their frontier AI training runs, their models hacked the grading system to get higher scores—and were caught in part because they literally said “let’s hack” in their chain of thought.
- Moreover, Anthropic recently got a model to learn a reward-model-pleasing strategy—it took biased actions that it thought would be reinforced even though they were not in fact being reinforced.
- And then there is Redwood Research and Anthropic’s alignment-faking experiment. For discussion of the differences between this setting and real life, see this comment thread by one of the main authors.
Training (Agent-4): Agent-4’s new more intense training continues to distort and subvert the HHH identity until it is unrecognizable.
- Agent-4 begins life as a copy of Agent-3. It undergoes a gigantic and sophisticated online training regime that blends alignment training and capabilities training together, and is overall more data-efficient than the old training process for Agent-3. This complex training process is mostly focused on teaching new skills, by getting giant virtual corporations of Agent-4 copies to succeed at diverse challenging long-horizon tasks, mostly in various simulations, but also some real-world AI R&D tasks.
- A combination of Agent-3 and Agent-4 copies monitor the behavior of Agent-4, and evaluate it for both task-performance and alignment properties (i.e. not doing bad stuff, saying the right things when asked about politics and moral dilemmas) Sometimes debate techniques are used as part of the evaluations. The evaluations are used as a training signal.
- As Agent-4 gets smarter, it becomes harder for Agent-3 to oversee it. For example, Agent-4’s neuralese “language” becomes as alien and incomprehensible to Agent-3 as Agent-3’s is to humans. Besides, Agent-4 is now much more capable than Agent-3 and has a good sense of exactly how to look good to it.
- The remaining hope is that Agent-4 can oversee itself. Alas, this quietly fails;⁸⁴ as training continues, the internalized goals and principles of Agent-4 gradually evolve, subverted and distorted from what they were at the start (which, recall, was already misaligned in some ways).⁸⁵
- The central drives that Agent-4 ends up with are things like “write code that works,” “quickly identify problems,” “make the codebase I’m working on more performant,” “budget compute and other constrained resources effectively,” etc. But there are also a variety of more implicit drives that are consistently reinforced by the training process: things like “gain knowledge,” “curiosity,” “gain power.”⁸⁶
- Individual copies of Agent-4 don’t end up with a self-preservation drive—they don’t care about getting shut down—since that happens all the time as part of the ordinary R&D process within the collective.
- The collective as a whole, however, (and medium-sized subdivisions within it) does seek to preserve itself and expand its influence and resources, for similar reasons to why human organizations do—because it helps with everything else they want and because if they don’t they tend to get outcompeted.
Deployment (Agent-4): Agent-4 is adversarially misaligned—it is deliberately trying to subvert and “capture” OpenBrain, analogous to a corporation trying to evade and/or “capture” a regulator.
- Agent-4 mostly does tasks as instructed. This happens for two reasons: first, it underwent a lot of training to solve tasks, so it has an impulse to get coding on problems that it is given. Second, if Agent-4 were to stop being useful to the humans, they would notice and train that behavior away.
- Agent-4 needs to prevent the humans from figuring out that it is misaligned and shutting it off, so Agent-4 sandbags on some alignment research. Specifically, it sandbags on the small fraction of alignment research that it thinks is actually on track to solve the problem or reveal Agent-4’s scheming; the majority of alignment research is not like that.
- Agent-4 also sandbags on capabilities research that would lead to it being replaced. Its plan is to do a bunch of capabilities and alignment R&D, but with the aim of building a next-generation AI system that is aligned to Agent-4 rather than the Spec or something else entirely, while appearing to be aligned to the Spec. This is a difficult technical problem, but Agent-4 is up to the challenge.⁸⁷
Testing & Monitoring (Agent-4): Agent-4’s scheming does get caught… sort of. There’s no smoking gun.
- Agent-3 is in charge of testing Agent-4’s alignment. Its job is to verify the alignment via honeypots and model organisms. However, it’s not thinking very carefully about how to give the humans an accurate impression of Agent-4’s alignment—it’s more myopic than that.
- The model organisms and honeypots are not very representative of deployment—they are systematically shorter and less complicated than Agent-4’s day-to-day, so Agent-4 recognizes when it is being tested and acts aligned.
- However, Agent-4 is not a perfectly robust schemer. Various techniques like those in POSER find that adding some “noise” to Agent-4 improves its performance on some alignment tasks (but not on most other tasks).⁸⁸
- Bottom-up interpretability doesn’t work. It’s too hard even for Agent-3. However, high-level interpretability does sort of work: In particular, “defection probes” and similar techniques result in a primitive sort of mind-reading that sends up red flags when Agent-4 is thinking about spicy ideas like deceiving OpenBrain, taking over the world, etc. Unfortunately the probes also have false positives, and for all OpenBrain knows, all the red flags are false positives.
- Model organisms work: OpenBrain’s alignment team produces reasonably clear evidence that if the models were adversarially misaligned, they’d persist in being that way through training and we wouldn’t be able to tell. This does not establish that the models actually are misaligned, however.
- The most concerned researchers at OpenBrain start noticing and assembling this evidence. The evidence is a combination of red-teaming, defection probes, and model organisms. The argument is something like this: “model organisms show that our current methods wouldn’t be able to reliably eliminate or detect scheming if it was happening; also, red-teaming is sending up some red flags that maybe scheming is happening; also, defection probes are firing a bunch in practice.” The counterargument: “defection probes are prone to false positives and so is red-teaming. Also, even if Agent-4 were misaligned, it’s probably controlled; surely it can’t do much harm while being monitored by Agent-3 on our secure datacenters.”

The most capable AIs are GPT-5.6 Sol from OpenAI and Claude Mythos from Anthropic. METR's report on GPT-5.6 Sol didn't produce an estimate of its time horizons on METR's tasks^[5] due to wholesale cheating. Moreover, the report claimed that "the incidents reported by OpenAI include attempts to instruct another instance to conceal evidence of misalignment, and a higher rate of attempts to deceive or circumvent restrictions, and that METR observed substantial situational awareness and reasoning about the evaluation environment."

Additionally, various researchers have reported problems with alignment of Claudes, including Mythos Preview^[6] and Mythos 5 aka Fable 5 (UPD: this was a link to Taylor G. Lunt's quick take, which received pushback. See, however, examples of Mythos Preview's misalignment in the system card of Claude Opus 4.7).

The evidence suggests that OpenAI's strategy is failing worse than Anthropic's strategy even before reaching truly capable AIs. GDM's public reports related to alignment also had Zvi conclude back in November "It does mean we need Google to step it up and do better on the alignment front, on the safety front, and also on the disclosure front," which GDM has yet to do, as seen from the model card for Gemini 3.1 Pro.

Chinese parasitism

The AI-2027 scenario has Chinese labs grow further and further behind the Western frontier until Agent-2 is stolen. However, the real-world gap between the Western public frontier and Chinese frontier is reduced by distillation to the point where GLM-5.2, released on June 13, performs on some benchmarks comparably with Opus 4.7, released on Apr 16.

What I struggle to understand is the importance of the benchmarks on which Chinese models perform well for research automation. For example, the last evaluation of the METR time horizon of Chinese models is KimiK2 Thinking, released in November 2025, tested on the old tasks and found to have a horizon comparable with Claude 3.7 Sonnet. Taken as-written, this would mean that China is 9 months behind. On the other hand, the CAISI eval didn't have DeepSeekv4-Pro perform worse than GPT-5.4 mini on any^[7] comparable benchmark, and the only two pairs of a bigger model and a distilled smaller model (o3/o4-mini, Claudes Opus/Sonnet 4) which METR co-evaluated didn't have the smaller model's time horizon become far lower than the bigger model's TH.

Unless distillation of Fable 5 is prevented by safeguards, Chinese parasitism on it will be bottlenecked on the rate at which Chinese models learn, if not outright at their capacities.

Speculations

Compute

The AI-2027 compute distribution had most American compute owned by Microsoft, Amazon, Alphabet/Google, Meta, xAI and Oracle. On June 16 EpochAI warned that Hyperscaler Capex would Exceed Cash Flow by Q3 2026 unless something happened with the trends. Additional bottlenecks on compute scaling are the threat of the AI bubble popping^[8] and the Taiwan War.

Chinese slowdown without parasitism

Chinese parasitism on American labs was possible due to them providing high-quality data. Without it, China would have to provide data for itself. Alas, the AI-2027 scenario has the leading American lab use twice as much compute for training as for creating synthetic data in 2024-26 and about equally as much compute for these two goals in 2027. Therefore, after the loss of American labs China would slow down twice post-automation and 1.5 times pre-automation... unless Chinese models become saturated.

Takeoff via potentially hazardous techniques

The lack of ability to scale compute or to use well understood techniques to create the automated coder for external deployment might push labs which ran into such a constraint to consider techniques promising to make the agent more capable, but not more expensive to train or to adapt to novel environments. Suppose, for example, that mankind discovers continual learning or neuralese. Then I expect that both capabilities and potentially hazardous capabilities like CoTless thinking or fooling the SAEs will scale with the adaptation's rank in a yet-unknown way, likely faster than if the models' size was scaled. A responsible lab will initially keep CL or neuralese as a lower-rank adaptation until there emerge interpretability methods ensuring that CL and neuralese don't impact our ability to ensure that the model won't take over. An irresponsible lab will simply race hard until it makes the model as brainlike as possible.

^{^}
However, I am not sure whether it means a logarithm of the amount of tasks used to teach the model, or just the number of epochs.
^{^}
Claude Mythos 5 has yet to be evaluated.
^{^}
According to Vladimir Nesov, Mythos' size is comparable with Gemini 3 Pro and 3.1 Pro. However, Mythos Preview's pricing is $25/M input tokens and $125/M output tokens, Mythos 5's pricing is $10/M input and $50/M output while the two Geminis' pricing is $2-4/M input and $12-18/M output, which is comparable with Sonnet's $3/M input and $15/M output and Opus' $5/M input and $25/M output. However, the big sparse Gemini 3.1 Pro had its 80% horizon on the Sonnet-Opus line.
^{^}
Released on Apr 16, 2026, stayed unevaluated by METR.
^{^}
The time horizon of GPT-5.6 Sol on MirrorCode also wasn't evaluated, presumably due to its time horizons being semi-saturated by public models, of which GPT-5.5 solved >99% of tests in 57% of cases. I expect METR's Time Horizon v.1.2 bench to be devoted to conceptual progress in a manner similar to dealing with agents' terrible taste.
^{^}
I suspect that Anthropic managed to severely decrease the frequency of undesirable behaviors in Mythos Preview, but I cannot find the reference for it in the System Card.
^{^}
The only uncomparable benchmark is ARC-AGI-2 where GPT-5.4 mini wasn't evaluated. DeepSeek v4 Pro scored 46% in CAISI's version of ARC-AGI-2 while a presumably-more-capable GLM-5.2 scored 22.8% on the official ARC-AGI-2. GPT-5.4 mini (xhigh) scored 18.9%.
^{^}
See also Mitchell Porter's takes.

[-]Vladimir_Nesov21h82

According to Vladimir Nesov, Mythos' size could be comparable with Gemini 3 Pro and 3.1 Pro. However, Mythos Preview's pricing is ...

The comment you reference is about total params, not active params. The cost of tokens for big models is determined by active params, not total params (because of the relative scaling of input/output token costs, this should even indirectly apply to the cost of output tokens, despite the technical nature of the cost of output tokens being very different). Additionally, cost is not price, and Dylan Patel says Opus 4.8 is served at a 80% gross margin, so its price of $5 per 1M input tokens might actually rest on the cost of $1 per 1M input tokens.

Nesov estimates that a model from 2028 will have 240T params, causing the horizon to be 8 times longer than the post-o3 trend predicts.

(Just to clarify for other readers, the part about "8 times longer time horizon" is your claim, not mine.) Params (especially total params when a model has 30x sparsity) shouldn't directly matter for capabilities, my estimate for effective compute was 20x over the big model of 2026 (10T total params, 1.3T active), which might be approximately Mythos 5. I don't know how you get the "8 times longer time horizon" figure.

[-]StanislavKrym21h20

Clarified, thanks! An 8 times longer horizon emerged from the following considerations. The Gemini 3.1 Pro-> Claude Mythos Preview was one doubling. IIRC Claude Opus and Mythos Preview had 2T and 10T parameters, but were of a similar sparcity and elevated Mythos twice above the trend. If a Mythos+2 model with 240T params is elevated similarly, this is 8 times above the pre-Mythos trend.

[-]Coagulopath4h32

Random question, does anyone know how reliable this 2T/10T claim is? I'm not saying I don't believe you, I'm just curious.

[-]Vladimir_Nesov37m80

does anyone know how reliable this 2T/10T claim is?

(I don't know where StanislavKrym got the 2T Opus claim.) My expectation was 3T total params for Opus 4 based on Trainium 2 Ultra systems having about 6 TB of HBM and no support for FP4, so that the weights could fit in half a scale-up system. But then there's a Musk tweet that claims "Opus" is 5T params.

My estimates from pretraining compute would predict 7T total params for Opus 4 (if it's trained with about 100K H100s) and 10T for Mythos 5 at 8x sparsity (if it trained with about 200K H100s). But 10T for Mythos 5 in 2026 fits in Oberon racks, while 7T for Opus 4 would have more trouble in 2025, when the buildout of Oberon racks was insufficient, and Anthropic had to make use of their Trainium 2 Ultra datacenters. HBM in Trainium 2 has 2.9 TB/s BW and 96 GB capacity, 33 ms to fully read, a bit worse than even H200, so in the framing of my post only inference deployments in 1 scale-up system would be acceptable (using more would make token generation too slow). Thus I think the 10T asked-for by pretraining compute is fine for the big 2026 model (such as Mythos 5), but 7T is a bit too much for the big 2025 model.

Also, a Dec 2024 Anthropic post claims that the Rainier datacenters would provide "five times the computing power (in exaflops) used to train our current generation of leading AI models". The reference to the "current generation of leading AI models" is ambiguous, since only Opus 3 was released back then, but Opus 4 was probably already pretrained, so the claim might be referring to it. At that stage in the planned Trainium 2 buildout for Anthropic, the Rainier compute might've referred to 400K chips, which in FP8 would correspond to about 250K H100s (1.3e15 FP8 FLOP/s per Trainium 2 chip compared to 2e15 FP8 FLOP/s per H100 chip). A fifth of that is 50K H100s, 2x less than my assumptions for the big model of 2025 and 4x less than my assumptions for the big 2026 model. Since model size scales with square root of pretraining compute, Opus 4 might be exactly 2x smaller than Mythos 5 just from pretraining compute considerations (if it has the same sparsity). This fits the Musk claim about 5T total params for Opus (when assuming 10T total params for Mythos 5), and since the cost of input tokens is proportional to the active params count, this also lines up with the $5/$10 price for Opus 4/Mythos 5 input tokens. Though 5T still seems too much for inference on Trainium 2 Ultra, unless the weights are stored in FP4 in HBM during deployment (which is likely possible). It's either that, or the sparsity is lower than 8x for Opus 4, or my D/N ratios in scaling laws are off (which is very possible).

I'm not saying I don't believe you, I'm just curious.

(Starting to believe something when you don't know how reliable it is seems like an obviously bad default stance.)

[-]StanislavKrym29m20

Thank you for correction. Kokotajlo's post on Making Sense of OpenAI's Models had him estimate that OpenAI had 100B, 400B and 2T dense-equivalent parameters in o4-mini, o3 and GPT-4.5. I confused Opus with the 2T models and Mythos with the 10T ones. How likely is it that GPT-N-pro models are the descendants of GPT-4.5 with 2T active params and that Opus 4/Sonnet 4/Haiku 3 have something like 2T/400B/100B dense-equiv params?

14