This is a special post for quick takes by Richard_Ngo. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
I previously noted that we’re seeing a pretty sharp divergence between the measured capabilities of models and their real-world impacts, which should suggest that something weird is going on.
A lot of people in this community are programmers and have the experience of the coding models having real-world impacts on their daily lives that aren't just about the models getting better at benchmarks.