Lots of people are trying to make AI good at AI research. How are they doing?
One way to measure this is to assume AIs are gradually doing more and more complex tasks independently. Eventually it would grow to doing whole research projects. Something like this, but for software engineering instead of research, is captured in the METR "time horizons" benchmark.
I think extending this line of thinking to forecasting progress in AI research is wrong. Instead, a better way to accelerate AI research for the time being is combining AI and people to do research together, in a way that uses the complementary strengths of each; with the goal of the researcher's feedback loops shortening.
What is difficult to automate in AI research?
If you decompose a big AI research project into tasks, there's lots of "dark matter" that does not neatly fit into any category. Some examples are given in Large-Scale Projects Stress Deep Cognitive Skills, which is a much better post than mine.
But I think that the most central argument is: the research process involves taste, coming up with ideas, and various such intangibles that we don't really know how to train for.
The labs are trying to make superhuman AI researchers. We do not yet know how to do it, which means at least some of our ideas are lacking. To improve our ideas, we need either:
Measuring which ideas are good is difficult; it requires sparse empirical outcomes that happen long after the idea is formulated. How can we accelerate this process?
I want to make two claims:
The careful reader might ask, ok, this sounds fine in the abstract, but I don't understand what exactly the lab is doing then, if not "automate AI research as a whole"? How is this different from making autonomous AI researchers directly?
Here is a list of tasks that would be extremely valuable if we wanted to make the research feedback loops faster.
I believe all of these tasks possess properties that make them attractive to attack directly.
This seems easier than automating the full research process. If labs have the goal of speeding up the lab's ability to do AI research as opposed to other goals, they are probably doing these things; and measuring the ability of AIs to do research autonomously is not going to give a good grasp on how quickly the lab is accelerating.