This is a special post for quick takes by emile delcourt. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
I think a lot about how to increase the chance of contributing something valuable to x-risk mitigation - and AI safety as my particular path. 5 weeks ago, @jaybailey suggested that one way to achieve that goal can be to Have an Unreasonably Specific Story about the Future (a Future That I Want, to be even more specific). Do I have a 5-minute story that helps to do this? I think yes: 1)The current environment and AI architecture incentivize continued growth in capabilities and scale, which also increase weaknesses and threats, and where agents are viewed as short-term profits and building trust at the expense of strategic resilience and rational safety. 2) Many stakeholders would lift their skepticism regarding the risks (and perhaps even advocate for a more cautious approach to AI in their own network) if presented with a compelling case of clear and present risk of irreversible damage to society, which would tilt the balance in favor of decisions that reduce the risk of catastrophic incidents 3) Creating such a case requires assessments of impacts and likelihoods that are a) accurate, high-elicitation, and up to date against the latest models, research, and threat intelligence. b) well-crafted to make immediate sense to the audience c) well publicized to reach the decision makers that matter 4) Each time I share a failure case / threat model / demo / paper, and a decision maker understands why it rises above the FUD, the alternative of stricter governance gains momentum. 5) When someone learns something, the next question is always the same. What do we do next? It's the golden moment. We organize behavior change, working extremely efficiently through chokepoints (even simplifying processes and systems to create some if necessary), to re-align systems, people, and impact. If doing this looks like a lower rate of acceleration, it is not slowing down, and if it is, that just indicates an opportunity to allocate resources more optimally - and we just show that.
The most valuable part of a story such as this, in my mind, is how it can act as a filter. Anything that does not meet your requirements can be disregarded as a line of work. From the original article: (https://www.lesswrong.com/posts/8t8jdTyq6X2B7D8St/have-an-unreasonably-specific-story-about-the-future)
"Anything that is unlikely to help with at least one of those critical paths is a strategy I reject as insufficient in my preference ordering."
So if we look at your story, it seems like the filtering line here is: "if presented with a compelling case of clear and present risk of irreversible damage to society".
So, what that means is, for every piece of persuasive material you're considering writing, you can now ask "Would this present a compelling case of clear and present risk of irreversible damage to society?" Ideally you'd have some ideas that can do all three of these things - if not, you may have to settle for two, but at least it provides something to filter against. And on the experiment level you can do similar. This time you have to ask "Is there a result I could reasonably get from this experiment that would help present a clear and present risk of irreversible damage?" because obviously you don't necessarily know the results in advance here. Though, in your shoes, I would recommend these further experiments only if you can't create this case yet, or if you have your best case already written up and now want to make it stronger. If your goal is to present this case, try to take the most direct path towards it that you can.
The other thing is that I think Step 5 isn't concrete enough yet. You say: "When someone learns something, the next question is always the same. What do we do next? It's the golden moment." I agree with you. However, if I am a politician asking you this question, and you reply with:
"We organize behavior change, working extremely efficiently through chokepoints (even simplifying processes and systems to create some if necessary), to re-align systems, people, and impact."
That's where you would lose me. What behavior do we need to change? What systems? What next steps can I, as a decision maker, take? I'd recommend coming up with some more actionable next steps to provide to those people who are convinced or open to your findings. If your golden moment is people asking you this exact question, you should be sure you're as ready as possible for people to do so.