This is a special post for quick takes by No77e. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
One difference between the releases of previous GPT versions and the release of GPT-5 is that it was clear that the previous versions were much bigger models trained with more compute than their predecessors. With the release of GPT-5, it's very unclear to me what OpenAI did exactly. If, instead of GPT-5, we had gotten a release that was simply an update of 4o + a new reasoning model (e.g., o4 or o5) + a router model, I wouldn't have been surprised by their capabilities. If instead GPT-4 were called something like GPT-3.6, we would all have been more or less equally impressed, no matter the naming. The number after "GPT" used to track something pretty specific that had to do with some properties of the base model, and I'm not sure it's still tracking the same thing now. Maybe it does, but it's not super clear from reading OpenAI's comms and from talking with the model itself. For example, it seems too fast to be larger than GPT-4.5.
Possibly an unlikely possibility, but could it be that different versions of GPT-5 (ie., normal model, thinking model, and thinking-pro model) are actually of different sizes? Or do we know for sure that they all share the same architecture?
4No77e
Ah, interesting! So the speed we see shouldn't tell us much about GPT-5's size.
I omitted one other factor from my shortform, namely cost. Do you think OpenAI would be willing to serve an 8T params (1T active) model for the price we're seeing? I'm basically trying to understand whether GPT-5 being served for relatively cheap should be a large or small update.
Prefill (processing of input tokens) is efficient, something like 60% compute utilization might be possible, and that only depends on the number of active params. Generation of output tokens is HBM bandwidth bound, depends on the number of total params and the number of KV cache sequences for requests in a batch that fit on the same system (which share the cost of chip-time[1]). With GB200 NVL72, batches could be huge, dividing the cost of output tokens (still probably several times more expensive per token than prefill).
For prefill, we can directly estima... (read more)