I built an (unpublished) TAI timelines model

I'd be excited to see this if it's substantially different from existing published models.

I account for potential coordinated delays, catastrophes, and a 15% chance that we're fundamentally wrong about all of this stuff.

+1 to noting this explicitly; everyone should distinguish between their conditional on no major disruptions and their unconditional models.

The Review for 2021 Review links that currently appear on reviews are broken.

The ideas in this post greatly influence how I think about AI timelines, and I believe they comprise the current single best way to forecast timelines.

A +12-OOMs-style forecast, like a bioanchors-style forecast, has two components:

1. an estimate of (effective) compute over time (including factors like compute getting cheaper and algorithms/ideas getting better in addition to spending increasing), and
2. a probability distribution on the (effective) training compute requirements for TAI (or equivalently the probability that TAI is achievable as a function of training compute).

Unlike bioanchors, a +12-OOMs-style forecast answers #2 by considering various kinds of possible transformative AI systems and using some combination of existing-system performance, scaling laws, principles, miscellaneous arguments, and inside-view intuition to how much compute they would require. Considering the "fun things" that could be built with more compute lets us use more inside-view knowledge than bioanchors-style analysis, while not committing to a particular path to TAI like roadmap-style analysis would.

In addition to introducing this forecasting method, this post has excellent analysis of some possible paths to TAI.

Hooray!

The cover of Reality & Reason should say "Book 1" not "Book 4".

Good post.

We also think that roughly 90% of (competent) community-builders are focused on “typical community-building” (designed to get more alignment researchers). We think that roughly 70% should do typical community-building, and 30% should be buying time.

Unless I missed it, you didn't discuss how community-builders can/should buy time. (If it's the same as technical researchers, it's weird to call them community-builders, I think.)

OK, and a thing-that-prevents-you-from-saying-"ah yes, that happened because a shard bid on it"-about-any-arbitrary-behavior is that shards are supposed to be reasonably farsighted/standard expected-utility-maximizers?

I... don't get what this post accomplishes. How can we test shard theory in reality? What does shard theory predict in reality? What kinds of facts about RL agents are predicted by shard theory?

Misc: [resolved]

This seems like a direct step down the CAIS path of development.

Why? Making language models better at physics instinctively feels like the general-agent-y path to me.

Interesting. Figure 2 looks pretty impressive. What do the phrases like "log(350) = 2.5B" at the bottom of the figure mean?

I can think of one very obvious explanation for these facts.

What is it? (Or: how could Sagbigsal be involved in cheating?)