Authors: Joshua Qin, Mohammad Khan, Jaray Liu

This blog post covers material for the sixth lecture of Harvard CS 2881r: AI Safety and Alignment, taught by Boaz Barak. We provide an outline of the lecture below.

Links

Introduction

The idea of recursive self-improvement lies at the core of contemporary debates over AI takeoff dynamics. As models become active participants in their own development pipelines, such as optimizing architectures, generating training data, and automating research, the feedback loop between capability and capability growth begins to close. Recent analyses, from Epoch’s Gate Model to Anthropic’s “Takeoff Speeds” and the Three Types of Intelligence Explosion framework, attempt to formalize this transition: when does scaling shift from incremental efficiency gains to endogenous acceleration? In this view, recursive self-improvement is not a speculative singularity but an emergent property of scaling laws, where performance improvement itself becomes the substrate of further progress. The frontier question, then, is quantitative rather than philosophical: what functional form best describes the curve of intelligence when the optimizer learns to optimize itself?

Intelligence, Formalized

We provide an economic viewpoint of intelligence. To formalize a notion of intelligence growth, we define an intelligence function $I (t)$ , which can measure say the length of duration that a human human would take to do tasks that AI can successfully complete within some fixed success rate, cost, and wall-clock time, e.g. 50% at $100 at an hour. We consider three options for growth of $I (t)$ :
1. Constant AI growth: $I (t + 1) = I (t) + c \to I (t) = c t + constant$
2. More intelligence = more growth : $I (t + 1) = c \cdot I (t), c > 1 \to I (t) = constant \cdot c^{t}$
3. More intelligence = more growth in less time: $I (t + \frac{d}{I (t)}) = c \cdot I (t) \to I (t) = \frac{constant}{t^{*} - t}$

This intelligence function serves as the foundation for production functions and models of automation. The third possibility, a super-exponential function, is most concerning, in which as intelligence increases, the time required for each improvement shrinks, reaching a vertical asymptote at $t = t^{*}$ . British mathematician I.J. Good described the phenomena as once an AI can autonomously improve its own design, “intelligence will increase rapidly until it fa...

Posts

Wikitag Contributions

Comments