TLDR: Skip to the last image at the bottom of my post to see the visualization. I don't usually post here, but I know you folks like forecasting and timelines, so I think you might appreciate this. The recent time horizons paper by METR is great, but it only visualized...
Addressing misconceptions about the big picture of AI progress. Why GPT-5 isn't late, why synthetic data is viable, and why the next 18 months of progress is likely to be greater than the last. This LW post is mainly summarizing my first blog post I've ever made (just published earlier...
Thanks for the comment.
In terms of the label noise I feel like that's decently accounted for already, as the calculations I'm using are actually not from specific model scores, but rather these points are sampled from me taking the intersecting points of a smoothed curve with the accuracy level for each of the models, and then I derive the factor that exists between that point and the 50% accuracy time horizon, as you can see here in the image at this link for example: https://prnt.sc/odMcvz0isuRU
Additionally, after I noted those factors for 50% to 80%, 90%, 95% and 99% individually for all 6 of those models, I averaged them across the models, which... (read more)
TLDR: Skip to the last image at the bottom of my post to see the visualization.
I don't usually post here, but I know you folks like forecasting and timelines, so I think you might appreciate this. The recent time horizons paper by METR is great, but it only visualized tasks that models could complete with 50-80% accuracy. I think it's highly useful, and arguably even more useful, to see when capabilities would hit 95% or 99% accuracy or beyond for various time horizons.
So I went through the paper with the goal of getting the information that will allow such a visualization to be created, and I found distributions shown of model-specific accuracy... (read 319 more words →)
Thanks, should be fixed now.