This is a special post for quick takes by Cameron D.. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Projects to improve AI Safety community epistemic infrastructure seem like high value to me.
The alignment problem seems pretty hard still, but it would feel one to several orders of magnitude harder without LW and all the epistemic infrastructure built up here. I'm talking about both the explicit infrastructure (wikitags, two-axis voting, recommendation algorithms), and implicit (norms and culture). The effort that the moderation team puts in is obviously a huge pillar too. By a naive induction it seems like other bets along this vector would be useful, whether large scope or small scope. Particularly weird/untested ideas. Could be complementary or orthogonal to existing infrastructure.
I've heard that there's about to be a large amount of money coming into AI Safety soon because of the Anthropic IPO. I'm pretty worried about that money being primarily dumped into highly correlated bets like 1,001 more LLM Safety projects. Maybe this could be a good way to diversify?
I don't think its a foregone conclusion that timelines are too short for efforts in this vein to be useful (pick one of: LLMs don't scale to AGI, we get a pause, LLMs scale to AGI but only slowly). And there are maybe even some worlds in which timelines get delayed (Idk, maybe we can better coordinate around a pause, or get one in the first place, or maintain one better once we get it)?
What's this being bottlenecked on right now? Talent or money? Or is there not actually much room left for improvement over what we have right now?
If AI continues to improve, AI-safety is going to go mainstream and the AI-safety community will become eventually 2-4 orders of magnitude larger. If possible you'd want to keep good epistemic norms in that world. The bay area AI-safety community although still influential will become a very small part of the overall safety community. Seems high value to me how to think about keeping good epistemic norms in this world.