romeo's Shortform

romeo

This is a special post for quick takes by romeo. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

Maybe i'm in an echo chamber or have just had my head in the sand while working on AI 2027, but now that i've been paying attention to AI safety for almost 2 years and seen my timelines gradually collapse, I really want to engage with compelling arguments that might lengthen my timelines again.

I have feel like there are a bunch of viewpoints expressed about long timelines/slow takeoff but a lack of arguments. This is me reaching out in the hope that people might point me to the best existing write-ups or maybe make new ones!

I am tracking things like: "takeoff will be slow because of experiment compute bottlenecks," or "timelines to AIs with good research taste are very long," or even more general "look how bad AI is at all this (not-super-relevant-to-a-software-only-singularity-)stuff that is so easy for humans!" but in my opinion, these are just viewpoints (which by the way, seem to often get stated very confidently in a way that makes me not trust the epistemology behind them). So sadly these statements don't tend to lengthen my timelines.

In my view, these viewpoints would become arguments if they were more like (excuse the spitballing):

"1e28 FLOPs of experiment compute is unlikely to produce much algorithmic progress + give a breakdown of why a compelling allocation of 1e28 FLOP doesn't get very far"
"Research taste is in a different reference class to the things AI has been making progress on recently + compelling reasoning, like, maybe:
- 'it has O(X) more degrees of freedom,'
- 'it has way less existing data and/or it's way harder to create data, or give a reward signal'
- 'things are looking grim for the likelihood of generalization to these kinds of skills'
"there are XYZ properties intelligence needed that can't be simulated by current hardware paradigms"

Currently I feel like I have a heavy tail on my timelines and takeoff speeds as a placeholder in lieu of arguments like this, that i'm hoping exist.

(I wrote up my long timelines take as a new post. It's somewhat tongue-in-cheek in only myopically making a few more legible points (for example, even slowly growing funding will match the 2,000x scaleout earlier than 2050 if compute price-performance continues on trend). But the overall framing is that there won't obviously be something left that's going to predictably blow up on a schedule if we survive 2028, and the danger of 2022-2028 will be matched only by the much more diluted danger of 2028-2050, with the usual basic progress in methods of AI train... (read more)

3Vladimir_Nesov1y

I wrote it up in a new shortform.

2Vladimir_Nesov1y

Burden of proof should follow value of information, not plausibility. In particular, the most profitable thing to pursue is arguments you understand less about, which is often arguments in favor of things you disbelieve (since you'd be already familiar with arguments that've previously convinced you). So if someone wants to convince you of something you already believe, then the burden of proof is on them, but not if they want to convince you of something you disbelieve and didn't get around to investigating yet.