hersheys

tl;dr: people should understand and think hard about the problems they work on. We’ve observed that those who work in AI safety (ourselves included) often rely on concerning heuristics when choosing what to work on. Running a conference is probably good, doing pragmatic alignment research might be good, and as long as such objectives don’t breach our internal models of what could contribute to reducing x-risk, these things are “what should be done”. But using such vibesy thought processes don’t always produce “actually impactful work” that would beat a prospective counterfactual. We wrote this post to share our observations and figure out what we should be doing instead. People don’t know what they’re working on AI safety is talent constrained. However, simply inflating the field doesn’t solve our bottleneck; rather, we need more people who understand the core arguments of AI safety. You can’t determine how to meaningfully contribute to AI safety without deeply knowing the problem you are trying to solve. Many newer people (us included!) rush into research, fellowships, and the like without building the context necessary for navigating the field. Agency-maxxing is not always good Moving fast is good. Moving too fast leads to poor ToC and sloppily executing projects. Many people working in AI safety seem to downweight spending time thinking about the actual problem they are trying to solve and backchaining from it, in favor of moving as fast as possible to get things done quick and dirty. While this enables faster execution and increases the volume of work being done, it doesn’t always move the needle on effectively reducing x-risk. The problem with force multipliers We categorize ToCs that focus on enhancing the impact of others as force multipliers (think upskilling, infrastructure work). There’s strong incentives to become a force multiplier: enhancing the impact of others through auxiliary work is (arguably) higher ROI and often more approachable than “dir