This is a special post for quick takes by Buck. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Man, I really feel like I'm having a crazy-pills moment with all the hype around the latest METR report.
From a previous comment I wrote (sorry for quoting this comment for the second time on this site):
If a researcher spends 2 hours a day doing real research and 6 hours doing easy bullshit, and you give them an AI that can do easy bullshit, now they can spend 8 hours doing real research. Congratulations, that's a 4x speedup! It doesn't necessarily imply the AI is anywhere close to being able to take over the remaining 2 hours and do the real research itself. It also doesn't not mean that.
I really don't get how you can interpret the METR report to imply that SAR/RSI are close. The interviews in the report are mostly researchers saying the AI models have taken over coding up the experiments (the easy part) so they can focus on the research (the hard part). E.g., "Now only bottlenecked on ideas & compute, no longer bottlenecked on code". This is a real speedup, but also doesn't indicate the AI models are on the verge of taking over the hard part!
RSI requires SAR (superhuman AI researcher). SAR is defined by AI 2027 as "an AI system that can do the job of the best human AI researcher but 30x faster and with 30x more agents." SC (superhuman coder) on the other hand, only requires the ability to do "coding tasks involved in AI research (e.g. experiment implementation but not ideation/prioritization)" 30x faster than human coders. SC is arguably close. But all this evidence in the METR report is evidence for SC happening soon, not SAR, no? I didn't see any part of the report that said, "Wow, AI is sure coming up with novel research areas all on its own, and developing theoretical improvements in our understanding of AI safety leading to new ideas humans hadn't thought of before." How can we have RSI without that?
Maybe you think the SC part is just the first step, and SAR will follow in the next 12 months, but people seem to have really strongly updated on the 2x uplift reported by METR, but how does that 2x uplift figure have any bearing on SAR? I guess I should mention that I'm operating from a frame where it's possible for AI to get really good at narrow tasks (coding) without that implying an equivalent improvement in all areas (general intelligence). This has been obviously possible since before the LLM days when we had superhuman narrow chess AI that couldn't do other tasks at all, and obviously happening since the early LLM days when LLMs showed "jagged" intelligence, for some reason being very good at some things while being totally idiotic at others. In particular, I think AI has seen the least progress at high-level, broad reasoning, the kind of reasoning that integrates information from different areas at once, and that's required for research (and the other hard tasks that our "borderline superhuman" AI fails at, like writing books anyone would want to read, generating social media posts that get a lot of attention, making a lot of money, generating new insights in science, writing funny jokes, and so on).