From the OpenAI post on research acceleration. A bunch of good data in there. This graph in particular caught my attention -- it seems to be measuring "In-practice horizon length" on real-world coding tasks happening at OpenAI.
The in-practice horizon length of OpenAI models seems to have gone from around 1-2h at the beginning of the year to something like 8h in July. So that's 2-3 doublings in half a year.
Also, the in-practice horizon length seems to be a bit less than the METR-measured horizon length, which was something like 5h at the start of the year (and is now unknown due to running out of long METR tasks).
No one told me to post this. And OpenAI is not a uniformly competent or particularly top-down organization. I'm like 95% sure this is some kind of simple mistake or weird design standard rather than a plot to confuse the AI Futures Project by putting confusing colors on the graph.
BTW the SVG file is downloadable and encodes the raw data, so you can point Claude at the blog post and make a graph with better colors easily.
3Marcus Plutowski
Not to sound flippant, but can't ChatGPT handle this sort of thing? It would be surprising that you'd need to go to a "design team" at this point in the game.