I just joined last week, and between my being cautious about acceleration, busy with onboarding, and most of the content being written already, I didn't contribute much to this post. But I'm happy to see this level of transparency from OpenAI. I hope it becomes standard in the future at OpenAI and at other companies (and AI safety shops, so we can see whether capabilities or safety is being accelerated more), and that our civilization will ultimately turn transparency into competent governance of the increasingly scary AI systems being developed.
I'm curious to hear which graphs people think are net positive vs net negative for OpenAI to have made and published, and what additional metrics would be useful. Hopefully this is just the start!
Tweet threads by others on the team:
Today we're releasing data on models accelerating research at OpenAI. Recursive self-improvement could be the most important contributor to AI capabilities over the next few years, but by default it will only be seen inside a few frontier AI labs. Being transparent is more urgent than ever, so we can inform the public discussion on whether and how to pace model development. I ask other AI companies to do the same.
Chris Ong (in this thread he goes through every plot in detail!)
Today we are releasing new data on how models have accelerated AI research at OpenAI since the start of this year. Recursive self-improvement will be one of the most consequential AI developments of the next few years, and a shared public understanding for how models enter into the AI research process will be a critical part of democratic governance of AI and pacing the model frontier.
At the current trajectory of capabilities growth, RSI could soon be one of the most consequential developments of our time. The prospect of rapid, unprecedented advances in machine intelligence call for us to have the most informed, transparent discussion we can have in service of the best interests of society.
Our new post shares an early snapshot of how models are contributing to research inside OpenAI, along with the methods behind our measurements - data that we hope will establish a norm of public disclosure & ground engaged discussions about what the pace & direction of research acceleration should look like.
Overall this is good and I'd want other labs to copy the format. Most of the graphs, though, measure inputs, not acceleration.
The most valuable section is 4. Restricting Astra cut its RL compute 59%, but other models picked up about 85% of the decline, so total RL compute barely moved. That's real governance evidence: model-specific restrictions barely slow aggregate progress unless they cover compute too. The intervention-rate breakdown is also good, because it's an honest counterweight to the success-rate curves.
The token spend ($600/day median, $7k p90), the 3.1 agent-workdays per... (130 more words)