I have been revisiting the 2023 post "AI Timelines" today. I would be interested in seeing what the participants within would offer as estimates at the present time.
From my own cursory estimation, it seems like Kokotajlo was wildly miscalibrated on many or most fronts, while Cotra and Erdil's predictions in 2023 seem fairly calibrated, leaning heavily in favor of Erdil (who seems in retrospect to be remarkably well-calibrated).
Scattershot takeaways:
Kokotajlo correctly assumed governments would completely fail to slow down timelines.
Erdil's predictions are interesting; he speaks like a gambler (in a very positive way); very precise, taking into account many systemic factors, while not allowing any one to dominate. Conservative revenue estimate in median prediction scenario for labs in 2030 (which seems reasonable given how historically unprecedented the scale of the current economic bubble has been), with seemingly-reasonable predictions across the board otherwise.
Cotra quote that I thought was fairly solid:
Yeah, I just think the way we get our OAI-engineer-replacing-thingie is going to be radically different cognitively than human OAI-engineers, in that it will have coding instincts honed through ancestral memory the way grizzly bears have salmon-catching instincts baked into them through their ancestral memory. For example, if you give it a body, I don't think it'd learn super quickly to catch antelope in the savannah, the way a baby human caveperson could learn to code if you transported them to today.
I think the ultimate conclusion I have is that habryka needs to do more interviews. He's good at them.
For what it's worth, I think my qualitative predictions in this essay were good, but because I was consistently putting 50% chance on the current path of AI scaling hitting limits, my "median world" looks less impressive than you might expect. I think I flagged this in the conversation - this "median" world I'm describing is basically "the current paradigm works, but barely".
I think the world we're actually in is more like my 75th percentile at that time (or, equivalently, my median conditioning on the current paradigm continuing to work well). So I think ... (read more)