Recursive self-improvement is already here.
This point is far from original. It’s been described before, for instance here, in Drexler’s Reframing Superintelligence, and (as I was working on this post) in Jack Clark’s newsletter and even by Yann LeCun. But sometimes I still hear people talk about preparing for “when recursive self-improvement kicks in,” implying that it hasn’t already.
The kinds of recursive self-improvement mentioned here aren’t exactly the frequently-envisioned scenario of a single AI system improving itself unencumbered. They instead rely on humans to make them work, and humans are inevitably slow and thus currently inhibit a discontinuous foom scenario.
It may be tempting to dismiss the kind of recursive self-improvement happening today as not real recursive self-improvement. To think about it as some future event that will start to happen that we need to prepare for. Yes, we need to prepare for increasing amounts of it, but it’s not in the future, it’s in the present.
Here are some currently existing examples (years given for the particular example linked):
- (2016) Models play against themselves in order to iteratively improve their performance in games, most notably in AlphaGo and its variants.
- (2016) Some neural architecture search techniques use one neural network to optimize the architectures of different neural networks.
- (2016) AI is being used to optimize data center cooling, helping reduce the cost of further scaling.
- (2021) Code generation tools like GitHub Copilot can be helpful to software engineers, including presumably some AI research engineers (anecdotally, I’ve found it helpful when doing engineering). Engineers may thus be faster at designing AI systems, including Copilot-like systems.
- (2021) Google uses deep reinforcement learning to optimize their AI accelerators.
- (2022) Neural networks, running on NVIDIA GPUs, have been used to design more efficient GPUs which can in turn run more neural networks.
- (2022) Neural networks are being used for compiler optimization in the popular LLVM compiler language, which Pytorch’s just-in-time compiler is based on.
Inspired by Victoria Krakovna’s specification gaming spreadsheet, I’ve made a spreadsheet here with these examples. Feel free to submit more here. I think the number of examples will continue to grow, making it useful to keep track of them.
If this feels underwhelming compared with the kinds of recursive self-improvement often written about, you’re right. But consider that the start of an exponential often feels underwhelming. As time goes on, I expect that humans will become less and less involved in the development of AI, with AI automating more and more of the process. This could very well feel sudden, but it won’t be unprecedented: it’s already begun.
I'm a bit confused by your response. First, the meat of the argument:
You are implicitly comparing two models: Mfast and Mslow, which make predictions about the world. Each model makes several claims, including the shape of the function governing AI improvement and about how the shape of that function comes about[1]. So far as I can tell, a typical central claim of people who endorse Mfast is that AIs working on themselves will allow their capabilities to grow hyper-exponentially. Those who endorse Mslow don't seem to dispute that self-improvement will occur, but expect it to be par for the course of a new technology and continue to be well modeled by exponential growth.
So, it seems to me that the existence of recursive self-improvement without an observed fast takeoff is evidence against Mfast. I presume you disagree, but I don't see how from a model selection framework. Mfast predicts either the data we observe now or a fast takeoff, whereas Mslow predicts only the exponential growth we are currently observing (do you disagree that we're in a time of exponential growth?). By the laws of probability, Mslow places higher probability on the current data than Mfast. Due to Bayes' rule, Mslow is therefore favored by the existing evidence (i.e. the Bayes factor indicates that you should update towards Mslow). Now, you might have a strong enough prior that you still favor Mfast, but if your model placed less probability mass on the current data than another model, you should update towards that other model.
Second (and lastly), a quibble:
Yitz's response uses the terms hard/soft takeoff, was that edited? Otherwise your argument against "continuous" as opposed to slow or soft comes off as a non-sequitor; that you're battling for terminological ground that isn't even under contention.
Different people will have different versions of each of these models. Some may even oscillate between them as is convenient for argumentative purposes (a-la motte and bailey).