Agent-Foundations Research Can Still Save Us, Even In Short-Timeline Worlds

I'm getting the feeling that the recent updates towards short timelines are coupled with updates towards considering working on hard theoretical problems of alignment even less promising than before. I don't think this is correct.

Historically, highly theoretical research had no short-term impact, because the period of crossing the theory-practice gap was measured in years. If that were still the case, then yes, short timelines would mean there'd be no time to implement any potential philosophical/theoretical breakthroughs even if they were found.

However, that's precisely what LLMs themselves change:

• They can, or will soon become, superhumanly fast at resolving arbitrary formally stated conjectures. This means that it would be possible to rapidly formalize any conceptual framework that you've made sufficient philosophical progress on. Once you're "in the neighbourhood of" the correct formalism, you'd be able to rapidly explore it by spamming possible conjecture formulations at LLMs, finding ones that work out, then hashing out the rest of it using those as an "anchor".
• They are, or will soon become, superhuman at arbitrary "workmanlike" math. This likely includes finding computationally tractable but formally valid approximations of high-level algorithms, comparing various alternate implementations of such algorithms for efficiency, et cetera.
• They are, or will soon become, superhuman at arbitrary software-engineering tasks.

Basically, if we assume that they'd be able to dramatically accelerate in-paradigm LLM research, that means they'd be able to dramatically accelerate any in-paradigm research. So once a paradigm is found – once the very basic high-level foundational theory is nailed down – LLMs would be able to take it the rest of the way to highly efficient practical implementations, down to high-performance scalable code. "Dual-use capabilities" cut both ways. (Conversely, if you don't believe that this will happen, you don't actually believe in near-term RSI.)

Note that this also doesn't require LLMs to be more aligned/trustworthy/faithful, or have better judgement, or better research taste, or be more steerable towards ill-specified objectives, et cetera. "More of the same", as far as AI progress goes, would get us there.

And that means there will, effectively, be no "theory-practice gap" anymore. Which would make various "alignment/agent-foundations moonshots" very much relevant short-term.

We can't offload the hard philosophical parts of our alignment homework to LLMs, but it appears we may be able to offload everything else.

(Personally, I think there's 50% chance that my agenda will reach the point of needed conceptual clarity within 2027, and 80% that I'd get there by EOY 2028. I've been making rapid progress; hopefully I'll be able to share some legible results within the coming months.)